2024-09-13T10:16:15.190 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T10:16:15.216 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T10:16:15.298 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352 branch: wip-aclamk-cbt-combined description: rados/thrash-old-clients/{0-distro$/{centos_9.stream} 0-size-min-size-overrides/2-size-2-min-size 1-install/reef backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} email: akupczyk@redhat.com first_in_suite: false job_id: '7903352' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'rbd object class functional tests ' name: akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 30 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-aclamk-cbt-combined ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 filestore odsync write: true osd backoff on degraded: true osd backoff on peering: true osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd max backfills: 2 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 0.5 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - \(MON_DOWN\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-aclamk-cbt-combined sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 owner: scheduled_akupczyk@teuthology priority: 160 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - - mon.c - osd.8 - osd.9 - osd.10 - osd.11 - client.2 seed: 5394 sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 sleep_before_teardown: 0 subset: 582/12160 suite: rados suite_branch: wip-aclamk-cbt-combined suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBn2M6kY71X6d/mKPZ9Y2qY6cRTZM51g5NzhoGzblP7YNyQgWkRQ/o6aNvo3HIpG7cwjSfMxNG9/A33ADyOVCwc= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/LpgsK7j/CmFRGp4ZJfrQV8R/Az2ZMtj82xSyor4Q8u9hjz0J+b1Dr3qY9pCaPXbV7e5w4F+TP4YTKPfzsLTk= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKSrvh4pAp8dgiYPhU34n0ej+PEsE2UB+eMv6dwZpDT1yC4evOT6tVYy1yQXg6IX37j7E86YBNPFsIvdBTLwgus= tasks: - install: branch: reef exclude_packages: - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - ceph-base-debuginfo - ceph-common-debuginfo - ceph-immutable-object-cache-debuginfo - ceph-radosgw-debuginfo - ceph-test-debuginfo - ceph-base-debuginfo - ceph-mgr-debuginfo - ceph-mds-debuginfo - ceph-mon-debuginfo - ceph-osd-debuginfo - ceph-fuse-debuginfo - librados-devel-debuginfo - libcephfs2-debuginfo - librados2-debuginfo - librbd1-debuginfo - python3-cephfs-debuginfo - python3-rados-debuginfo - python3-rbd-debuginfo - python3-rgw-debuginfo - rbd-fuse-debuginfo - rbd-mirror-debuginfo - rbd-nbd-debuginfo - ceph-volume - cephadm: conf: mon: auth allow insecure global id reclaim: true - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.2: - ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_09:37:56 tube: smithi user: akupczyk verbose: true verify_ceph_hash: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T10:16:15.298 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa; will attempt to use it 2024-09-13T10:16:15.299 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa/tasks 2024-09-13T10:16:15.299 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T10:16:15.300 INFO:teuthology.task.internal:Saving configuration 2024-09-13T10:16:15.314 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T10:16:15.326 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T10:16:15.350 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 10:11:17.896564', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBn2M6kY71X6d/mKPZ9Y2qY6cRTZM51g5NzhoGzblP7YNyQgWkRQ/o6aNvo3HIpG7cwjSfMxNG9/A33ADyOVCwc='} 2024-09-13T10:16:15.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 10:11:17.897991', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/LpgsK7j/CmFRGp4ZJfrQV8R/Az2ZMtj82xSyor4Q8u9hjz0J+b1Dr3qY9pCaPXbV7e5w4F+TP4YTKPfzsLTk='} 2024-09-13T10:16:15.391 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 10:11:17.899204', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKSrvh4pAp8dgiYPhU34n0ej+PEsE2UB+eMv6dwZpDT1yC4evOT6tVYy1yQXg6IX37j7E86YBNPFsIvdBTLwgus='} 2024-09-13T10:16:15.391 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T10:16:15.399 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-13T10:16:15.399 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-09-13T10:16:15.399 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.c', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'client.2'] 2024-09-13T10:16:15.399 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T10:16:15.513 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8af5e07be0>, signals=[15]) 2024-09-13T10:16:15.514 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T10:16:15.520 INFO:teuthology.task.internal:Opening connections... 2024-09-13T10:16:15.520 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T10:16:15.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:16:15.598 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-09-13T10:16:15.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:16:15.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-09-13T10:16:15.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:16:15.749 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T10:16:15.756 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-09-13T10:16:15.774 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-09-13T10:16:15.774 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-09-13T10:16:15.831 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-09-13T10:16:15.832 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T10:16:15.832 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-09-13T10:16:15.832 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T10:16:15.832 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T10:16:15.832 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T10:16:15.832 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-09-13T10:16:15.848 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-09-13T10:16:15.867 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-09-13T10:16:15.868 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-09-13T10:16:15.926 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T10:16:15.927 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T10:16:15.928 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T10:16:15.928 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-09-13T10:16:15.949 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-09-13T10:16:15.967 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-09-13T10:16:15.967 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-09-13T10:16:16.025 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T10:16:16.026 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T10:16:16.027 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T10:16:16.027 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-09-13T10:16:16.057 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T10:16:16.067 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T10:16:16.128 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T10:16:16.128 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-09-13T10:16:16.131 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-09-13T10:16:16.134 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-09-13T10:16:16.151 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T10:16:16.157 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T10:16:16.157 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-09-13T10:16:16.188 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-09-13T10:16:16.191 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-09-13T10:16:16.296 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T10:16:16.372 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T10:16:16.387 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T10:16:16.387 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T10:16:16.388 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T10:16:16.388 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T10:16:16.389 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T10:16:16.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:16:16.406 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-13T10:16:16.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:16:16.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:16:16.408 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-09-13T10:16:16.408 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-09-13T10:16:16.464 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-503.el9.x86_64 2024-09-13T10:16:16.464 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-503.el9.x86_64 2024-09-13T10:16:16.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2024-09-13T10:16:16.465 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-503.el9.x86_64 2024-09-13T10:16:16.466 INFO:teuthology.task.kernel:Running kernel on smithi173: 5.14.0-503.el9.x86_64 2024-09-13T10:16:16.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum install -y kernel 2024-09-13T10:16:16.470 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-503.el9.x86_64 2024-09-13T10:16:16.470 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-503.el9.x86_64 2024-09-13T10:16:16.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-09-13T10:16:18.798 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-13T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-13T10:16:20.341 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:02 2024-09-13T10:16:20.541 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-13T10:16:24.225 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 20 MB 00:02 2024-09-13T10:16:27.641 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-09-13T10:16:28.468 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-13T10:16:29.367 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-13T10:16:29.935 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-13T10:16:30.146 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-13T10:16:30.739 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-13T10:16:33.248 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 20 MB 00:12 2024-09-13T10:16:35.494 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-13T10:16:36.203 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-13T10:16:37.500 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T10:16:37.500 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T10:16:37.589 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T10:16:37.596 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:16:37.597 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 79 M 2024-09-13T10:16:37.598 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 126 M 2024-09-13T10:16:37.598 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:16:37.912 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 78 kB/s | 23 kB 00:00 2024-09-13T10:16:38.239 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T10:16:38.239 INFO:teuthology.orchestra.run.smithi173.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T10:16:38.327 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:16:38.336 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T10:16:38.337 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:16:38.338 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:16:38.338 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:16:38.338 INFO:teuthology.orchestra.run.smithi173.stdout:Install 4 Packages 2024-09-13T10:16:38.338 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:16:38.338 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 79 M 2024-09-13T10:16:38.338 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 126 M 2024-09-13T10:16:38.339 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:16:38.771 INFO:teuthology.orchestra.run.smithi173.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 71 kB/s | 23 kB 00:00 2024-09-13T10:16:39.140 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-13T10:16:39.820 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.1 MB/s | 16 MB 00:02 2024-09-13T10:16:40.880 INFO:teuthology.orchestra.run.smithi173.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 14 MB/s | 35 MB 00:02 2024-09-13T10:16:40.907 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-13T10:16:41.487 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 8.0 MB/s | 29 MB 00:03 2024-09-13T10:16:41.703 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-13T10:16:41.838 INFO:teuthology.orchestra.run.smithi173.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 9.3 MB/s | 29 MB 00:03 2024-09-13T10:16:46.520 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-13T10:16:46.520 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:16:46.520 INFO:teuthology.orchestra.run.smithi138.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-13T10:16:47.195 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:16:47.418 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-13T10:16:47.562 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:16:47.562 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:16:48.523 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:16:48.524 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:16:49.004 INFO:teuthology.orchestra.run.smithi173.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.5 MB/s | 16 MB 00:10 2024-09-13T10:16:49.004 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:16:49.005 INFO:teuthology.orchestra.run.smithi173.stdout:Total 7.4 MB/s | 79 MB 00:10 2024-09-13T10:16:49.508 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T10:16:49.508 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T10:16:49.601 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T10:16:49.607 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T10:16:49.608 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:16:49.608 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:16:49.608 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:16:49.608 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2024-09-13T10:16:49.608 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:16:49.609 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 79 M 2024-09-13T10:16:49.609 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 126 M 2024-09-13T10:16:49.609 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:16:49.690 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:16:49.961 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 235 kB/s | 23 kB 00:00 2024-09-13T10:16:50.029 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:16:50.029 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:16:50.394 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-13T10:16:50.828 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-13T10:16:50.955 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:16:51.038 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:16:51.039 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:16:51.387 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-13T10:16:51.387 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:16:51.388 INFO:teuthology.orchestra.run.smithi060.stdout:Total 44 MB/s | 79 MB 00:01 2024-09-13T10:16:51.615 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T10:16:51.697 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:16:52.101 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:16:52.511 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:16:52.511 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:16:53.467 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:16:53.526 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:16:53.527 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:16:54.305 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T10:16:54.396 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:16:54.505 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:16:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:16:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:16:56.808 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T10:16:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:16:57.195 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:16:57.336 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:16:59.446 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:16:59.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:16:59.699 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:16:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:17:02.194 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:17:02.319 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:04.428 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:17:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:07.190 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:10.132 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:42.863 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:42.903 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:43.219 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:43.219 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T10:17:43.219 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:17:43.219 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T10:17:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T10:17:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:17:44.125 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:17:44.492 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T10:17:44.574 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:44.574 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T10:17:44.574 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T10:17:44.574 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T10:17:44.575 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T10:17:44.575 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T10:17:44.575 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T10:17:44.575 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T10:17:44.633 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:44.634 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T10:17:44.634 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-09-13T10:17:44.665 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-09-13T10:17:44.665 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-09-13T10:17:44.665 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 82.el9 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8274670 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-09-13T10:17:44.666 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:17:44.667 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T10:17:44.668 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T10:17:44.668 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 5.14.0-505.el9.x86_64 2024-09-13T10:17:44.668 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T10:17:45.206 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:45.246 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:45.315 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-09-13T10:17:45.608 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:45.608 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T10:17:45.608 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:17:45.608 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:17:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:17:46.432 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:17:46.614 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T10:17:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-09-13T10:17:46.653 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-09-13T10:17:46.669 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.TRrMU6QDiW 2024-09-13T10:17:46.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TRrMU6QDiW 2024-09-13T10:17:46.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.TRrMU6QDiW 2024-09-13T10:17:46.805 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T10:17:46.884 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.TRrMU6QDiW is 6KB 2024-09-13T10:17:46.893 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:46.893 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T10:17:46.894 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T10:17:46.894 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T10:17:46.894 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T10:17:46.894 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-13T10:17:46.894 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi173.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T10:17:46.894 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T10:17:46.932 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.TRrMU6QDiW 2024-09-13T10:17:46.947 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T10:17:46.951 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:46.952 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T10:17:46.952 DEBUG:teuthology.orchestra.run.smithi173:> sudo rpm -qi grub2-tools 2024-09-13T10:17:46.984 INFO:teuthology.orchestra.run.smithi173.stdout:Name : grub2-tools 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Epoch : 1 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Version : 2.06 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Release : 82.el9 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: x86_64 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Group : Unspecified 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Size : 8274670 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:License : GPLv3+ 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T10:17:46.985 INFO:teuthology.orchestra.run.smithi173.stdout:Packager : builder@centos.org 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:Vendor : CentOS 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:Summary : Support tools for GRUB. 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:Description : 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:hardware devices. 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:17:46.986 INFO:teuthology.orchestra.run.smithi173.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T10:17:46.987 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T10:17:46.987 INFO:teuthology.task.kernel:Updating grub on smithi173 to boot 5.14.0-505.el9.x86_64 2024-09-13T10:17:46.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T10:17:47.010 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T10:17:47.010 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T10:17:47.010 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T10:17:47.010 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T10:17:47.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T10:17:47.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-09-13T10:17:47.568 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-09-13T10:17:48.915 INFO:teuthology.orchestra.run.smithi173.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T10:17:48.947 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-09-13T10:17:48.949 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-09-13T10:17:48.966 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.YFf9fu1Yyr 2024-09-13T10:17:48.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YFf9fu1Yyr 2024-09-13T10:17:49.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.YFf9fu1Yyr 2024-09-13T10:17:49.176 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.YFf9fu1Yyr is 6KB 2024-09-13T10:17:49.225 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.YFf9fu1Yyr 2024-09-13T10:17:49.240 DEBUG:teuthology.orchestra.run.smithi173:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T10:17:49.275 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:49.303 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T10:17:49.303 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T10:17:49.303 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T10:17:49.303 INFO:teuthology.orchestra.run.smithi173.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T10:17:49.304 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T10:17:49.316 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:49.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo shutdown -r now 2024-09-13T10:17:49.766 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:49.766 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T10:17:49.766 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T10:17:49.767 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T10:17:50.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T10:17:50.627 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:17:50.627 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:17:50.628 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:50.628 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T10:17:50.628 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T10:17:50.628 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T10:17:50.628 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:17:50.628 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:17:50.974 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T10:17:51.049 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:51.050 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T10:17:51.050 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T10:17:51.051 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T10:17:51.051 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T10:17:51.051 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T10:17:51.051 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T10:17:51.051 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T10:17:51.109 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T10:17:51.109 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T10:17:51.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-09-13T10:17:51.144 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-09-13T10:17:51.144 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 82.el9 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8274670 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-09-13T10:17:51.145 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:17:51.146 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T10:17:51.147 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T10:17:51.147 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 5.14.0-505.el9.x86_64 2024-09-13T10:17:51.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T10:17:51.777 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-09-13T10:17:53.077 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T10:17:53.101 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-09-13T10:17:53.103 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-13T10:17:53.118 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.lm45m2BBLH 2024-09-13T10:17:53.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lm45m2BBLH 2024-09-13T10:17:53.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.lm45m2BBLH 2024-09-13T10:17:53.326 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.lm45m2BBLH is 6KB 2024-09-13T10:17:53.346 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.lm45m2BBLH 2024-09-13T10:17:53.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T10:17:53.429 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T10:17:53.429 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T10:17:53.429 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T10:17:53.429 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T10:17:53.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T10:17:53.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-09-13T10:18:17.213 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T10:18:17.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-13T10:18:17.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:18:19.461 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-13T10:18:19.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:18:19.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:18:23.585 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T10:18:23.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T10:18:23.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:18:35.713 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-09-13T10:18:38.013 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-13T10:18:44.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-13T10:18:44.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:18:47.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:18:47.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:18:47.805 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-09-13T10:18:50.081 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-13T10:18:59.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-13T10:18:59.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:19:02.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:19:02.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:19:20.509 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-13T10:19:23.590 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T10:19:31.717 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-13T10:19:32.312 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-13T10:19:32.312 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T10:19:32.313 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-09-13T10:19:32.350 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-505.el9.x86_64 2024-09-13T10:19:32.351 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T10:19:32.351 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T10:19:32.351 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T10:19:32.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T10:19:32.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:19:33.190 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-13T10:19:33.352 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T10:19:33.353 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T10:19:33.526 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T10:19:33.526 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T10:19:33.527 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-13T10:19:33.543 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-505.el9.x86_64 2024-09-13T10:19:33.543 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T10:19:33.543 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T10:19:33.543 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T10:19:33.740 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-09-13T10:19:33.770 DEBUG:teuthology.parallel:result is None 2024-09-13T10:19:34.544 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T10:19:34.545 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T10:19:34.653 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2024-09-13T10:19:34.683 DEBUG:teuthology.parallel:result is None 2024-09-13T10:19:35.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:19:35.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:19:35.843 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-13T10:19:36.231 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:19:36.231 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-13T10:19:36.232 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-09-13T10:19:36.251 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-505.el9.x86_64 2024-09-13T10:19:36.252 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T10:19:36.252 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T10:19:36.252 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T10:19:37.253 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-13T10:19:37.253 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T10:19:37.358 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2024-09-13T10:19:37.389 DEBUG:teuthology.parallel:result is None 2024-09-13T10:19:37.389 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T10:19:37.402 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T10:19:37.402 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T10:19:37.405 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T10:19:37.408 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T10:19:37.465 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T10:19:37.476 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T10:19:37.564 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T10:19:37.564 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T10:19:37.567 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T10:19:37.570 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T10:19:37.636 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T10:19:37.664 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T10:19:37.665 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T10:19:37.667 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T10:19:37.669 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T10:19:37.696 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T10:19:37.706 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T10:19:37.708 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T10:19:37.717 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T10:19:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T10:19:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T10:19:37.756 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T10:19:37.767 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T10:19:37.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T10:19:37.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T10:19:37.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T10:19:37.846 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T10:19:37.869 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T10:19:37.869 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T10:19:37.872 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T10:19:37.874 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T10:19:37.920 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T10:19:37.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T10:19:38.028 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T10:19:38.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T10:19:38.166 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:19:38.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T10:19:38.260 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T10:19:38.309 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T10:19:38.360 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T10:19:38.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T10:19:38.495 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:19:38.495 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T10:19:38.563 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T10:19:38.608 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T10:19:38.673 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T10:19:38.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T10:19:38.756 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:19:38.757 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T10:19:38.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-09-13T10:19:38.825 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-09-13T10:19:38.827 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-09-13T10:19:38.867 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T10:19:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T10:19:38.904 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T10:19:39.394 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T10:19:39.401 INFO:teuthology.task.internal:Starting timer... 2024-09-13T10:19:39.402 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T10:19:39.449 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T10:19:39.516 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-13T10:19:39.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-09-13T10:19:39.592 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-09-13T10:19:39.594 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-09-13T10:19:39.662 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-09-13T10:19:39.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-09-13T10:19:39.724 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-09-13T10:19:39.726 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T10:19:39.726 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-09-13T10:19:39.754 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-09-13T10:19:39.754 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-09-13T10:19:39.793 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-09-13T10:19:39.794 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-09-13T10:19:39.817 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-09-13T10:19:39.818 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2024-09-13T10:19:39.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T10:19:39.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:19:39.849 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T10:19:39.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:19:39.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T10:19:39.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:19:39.922 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T10:19:39.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-09-13T10:19:39.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-09-13T10:19:39.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-09-13T10:19:40.029 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T10:19:40.042 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T10:19:40.045 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T10:19:40.167 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T10:19:40.178 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T10:19:40.179 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi138.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T10:23:30.728 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-09-13T10:23:30.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T10:23:30.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:23:30.810 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-13T10:23:30.889 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T10:23:30.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-13T10:23:30.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:23:30.973 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-13T10:23:31.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-13T10:23:31.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:23:31.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:23:31.121 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-13T10:23:31.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-13T10:23:31.198 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T10:23:31.209 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T10:23:31.209 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:23:31.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:23:31.212 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:23:31.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:23:31.215 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:23:31.215 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:23:31.243 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T10:23:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T10:23:31.258 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T10:23:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T10:23:31.268 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T10:23:31.283 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-09-13T10:23:31.283 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T10:23:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-09-13T10:23:31.294 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-09-13T10:23:31.299 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-09-13T10:23:31.308 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-09-13T10:23:31.309 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T10:23:31.316 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T10:23:31.319 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-09-13T10:23:31.322 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T10:23:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T10:23:31.333 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T10:23:31.347 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T10:23:31.374 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-13T10:23:31.377 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T10:23:31.377 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-13T10:23:31.378 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.378 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.378 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.378 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-13T10:23:31.394 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T10:23:31.394 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-13T10:23:31.395 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.395 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.395 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.395 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.400 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-09-13T10:23:31.404 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T10:23:31.404 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-09-13T10:23:31.404 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.404 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.404 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.404 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:23:31.405 INFO:teuthology.run_tasks:Running task install... 2024-09-13T10:23:31.412 DEBUG:teuthology.task.install:project ceph 2024-09-13T10:23:31.413 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}} 2024-09-13T10:23:31.413 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo', 'ceph-volume'], 'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'} 2024-09-13T10:23:31.413 INFO:teuthology.task.install:Using flavor: default 2024-09-13T10:23:31.418 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T10:23:31.419 INFO:teuthology.task.install:extra packages: [] 2024-09-13T10:23:31.419 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo', 'ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'tag': None, 'wait_for_package': False} 2024-09-13T10:23:31.419 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:23:31.419 INFO:teuthology.packaging:ref: None 2024-09-13T10:23:31.420 INFO:teuthology.packaging:tag: None 2024-09-13T10:23:31.420 INFO:teuthology.packaging:branch: reef 2024-09-13T10:23:31.420 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:23:31.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T10:23:31.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo', 'ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'tag': None, 'wait_for_package': False} 2024-09-13T10:23:31.422 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:23:31.422 INFO:teuthology.packaging:ref: None 2024-09-13T10:23:31.422 INFO:teuthology.packaging:tag: None 2024-09-13T10:23:31.423 INFO:teuthology.packaging:branch: reef 2024-09-13T10:23:31.423 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:23:31.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T10:23:31.424 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo', 'ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'tag': None, 'wait_for_package': False} 2024-09-13T10:23:31.424 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:23:31.424 INFO:teuthology.packaging:ref: None 2024-09-13T10:23:31.424 INFO:teuthology.packaging:tag: None 2024-09-13T10:23:31.424 INFO:teuthology.packaging:branch: reef 2024-09-13T10:23:31.424 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:23:31.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T10:23:31.664 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/ 2024-09-13T10:23:31.665 INFO:teuthology.task.install.rpm:Package version is 18.2.4-843.g2c01458a 2024-09-13T10:23:31.681 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/ 2024-09-13T10:23:31.682 INFO:teuthology.task.install.rpm:Package version is 18.2.4-843.g2c01458a 2024-09-13T10:23:31.686 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/ 2024-09-13T10:23:31.686 INFO:teuthology.task.install.rpm:Package version is 18.2.4-843.g2c01458a 2024-09-13T10:23:31.779 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T10:23:31.779 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:23:31.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T10:23:31.798 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T10:23:31.798 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:23:31.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T10:23:31.801 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/2c01458affee2b2435cd231b6e392435c1c21661/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T10:23:31.801 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:23:31.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T10:23:31.805 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T10:23:31.805 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:23:31.805 INFO:teuthology.packaging:ref: None 2024-09-13T10:23:31.805 INFO:teuthology.packaging:tag: None 2024-09-13T10:23:31.806 INFO:teuthology.packaging:branch: reef 2024-09-13T10:23:31.806 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:23:31.806 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T10:23:31.826 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T10:23:31.826 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:23:31.827 INFO:teuthology.packaging:ref: None 2024-09-13T10:23:31.827 INFO:teuthology.packaging:tag: None 2024-09-13T10:23:31.827 INFO:teuthology.packaging:branch: reef 2024-09-13T10:23:31.827 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:23:31.827 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T10:23:31.830 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T10:23:31.830 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:23:31.830 INFO:teuthology.packaging:ref: None 2024-09-13T10:23:31.831 INFO:teuthology.packaging:tag: None 2024-09-13T10:23:31.831 INFO:teuthology.packaging:branch: reef 2024-09-13T10:23:31.831 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:23:31.831 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T10:23:31.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T10:23:31.893 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T10:23:31.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T10:23:31.946 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T10:23:31.968 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T10:23:31.979 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T10:23:32.011 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-09-13T10:23:32.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-09-13T10:23:32.032 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-09-13T10:23:32.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-09-13T10:23:32.045 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-09-13T10:23:32.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-09-13T10:23:32.241 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-09-13T10:23:32.269 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-09-13T10:23:32.273 INFO:teuthology.orchestra.run.smithi173.stdout:45 files removed 2024-09-13T10:23:32.296 INFO:teuthology.orchestra.run.smithi060.stdout:45 files removed 2024-09-13T10:23:32.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw 2024-09-13T10:23:32.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw 2024-09-13T10:23:32.871 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 245 kB/s | 77 kB 00:00 2024-09-13T10:23:32.887 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 258 kB/s | 77 kB 00:00 2024-09-13T10:23:32.902 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 275 kB/s | 77 kB 00:00 2024-09-13T10:23:33.163 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T10:23:33.188 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-13T10:23:33.203 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-13T10:23:33.439 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T10:23:33.463 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T10:23:33.479 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-13T10:23:34.163 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-13T10:23:34.888 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.2 MB 00:01 2024-09-13T10:23:35.122 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-13T10:23:36.723 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-13T10:23:37.122 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-13T10:23:38.591 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 20 MB 00:02 2024-09-13T10:23:41.249 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-13T10:23:42.414 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.5 MB 00:01 2024-09-13T10:23:42.791 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-13T10:23:42.833 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-13T10:23:43.695 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-13T10:23:44.182 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-13T10:23:44.518 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-13T10:23:44.944 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-13T10:23:45.306 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-13T10:23:49.276 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-13T10:23:50.625 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-13T10:23:50.712 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-13T10:23:51.634 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:23:51.645 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:23:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-13T10:23:51.645 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:23:51.645 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:23:51.645 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:18.2.4-843.g2c01458a.el9 ceph 7.7 M 2024-09-13T10:23:51.645 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.3 M 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.0 M 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:18.2.4-843.g2c01458a.el9 ceph 5.1 M 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:18.2.4-843.g2c01458a.el9 ceph 18 M 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:18.2.4-843.g2c01458a.el9 ceph 25 k 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T10:23:51.646 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 712 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 477 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 4.5 M 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 45 k 2024-09-13T10:23:51.647 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:18.2.4-843.g2c01458a.el9 ceph 129 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:18.2.4-843.g2c01458a.el9 ceph 162 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:18.2.4-843.g2c01458a.el9 ceph 321 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 301 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:18.2.4-843.g2c01458a.el9 ceph 99 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout:Install 28 Packages 2024-09-13T10:23:51.648 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 2 Packages 2024-09-13T10:23:51.649 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:23:51.649 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 53 M 2024-09-13T10:23:51.650 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:23:52.335 INFO:teuthology.orchestra.run.smithi173.stdout:(1/30): ceph-base-18.2.4-843.g2c01458a.el9.x86_ 9.7 MB/s | 5.1 MB 00:00 2024-09-13T10:23:52.886 INFO:teuthology.orchestra.run.smithi173.stdout:(2/30): ceph-common-18.2.4-843.g2c01458a.el9.x8 17 MB/s | 18 MB 00:01 2024-09-13T10:23:52.911 INFO:teuthology.orchestra.run.smithi173.stdout:(3/30): ceph-selinux-18.2.4-843.g2c01458a.el9.x 44 kB/s | 25 kB 00:00 2024-09-13T10:23:53.019 INFO:teuthology.orchestra.run.smithi173.stdout:(4/30): libcephfs2-18.2.4-843.g2c01458a.el9.x86 5.2 MB/s | 712 kB 00:00 2024-09-13T10:23:53.078 INFO:teuthology.orchestra.run.smithi173.stdout:(5/30): libradosstriper1-18.2.4-843.g2c01458a.e 2.8 MB/s | 477 kB 00:00 2024-09-13T10:23:53.105 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:18.2.4-843.g2c01458a.el9 ceph 7.7 M 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.3 M 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.0 M 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-13T10:23:53.119 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:18.2.4-843.g2c01458a.el9 ceph 5.1 M 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:18.2.4-843.g2c01458a.el9 ceph 18 M 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:18.2.4-843.g2c01458a.el9 ceph 25 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 712 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T10:23:53.120 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 477 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 4.5 M 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 45 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:18.2.4-843.g2c01458a.el9 ceph 129 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:18.2.4-843.g2c01458a.el9 ceph 162 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:18.2.4-843.g2c01458a.el9 ceph 321 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 301 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:18.2.4-843.g2c01458a.el9 ceph 99 k 2024-09-13T10:23:53.121 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout:Install 28 Packages 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:23:53.122 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 53 M 2024-09-13T10:23:53.123 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:23:53.238 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:18.2.4-843.g2c01458a.el9 ceph 7.7 M 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.3 M 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.0 M 2024-09-13T10:23:53.253 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:18.2.4-843.g2c01458a.el9 ceph 5.1 M 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:18.2.4-843.g2c01458a.el9 ceph 18 M 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:18.2.4-843.g2c01458a.el9 ceph 25 k 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 712 k 2024-09-13T10:23:53.254 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 477 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:18.2.4-843.g2c01458a.el9 ceph 4.5 M 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 45 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:18.2.4-843.g2c01458a.el9 ceph 129 k 2024-09-13T10:23:53.255 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:18.2.4-843.g2c01458a.el9 ceph 162 k 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:18.2.4-843.g2c01458a.el9 ceph 321 k 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 301 k 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:18.2.4-843.g2c01458a.el9 ceph 99 k 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:23:53.256 INFO:teuthology.orchestra.run.smithi060.stdout:Install 28 Packages 2024-09-13T10:23:53.257 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 2 Packages 2024-09-13T10:23:53.257 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:23:53.257 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 53 M 2024-09-13T10:23:53.257 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:23:53.503 INFO:teuthology.orchestra.run.smithi173.stdout:(6/30): librgw2-18.2.4-843.g2c01458a.el9.x86_64 9.2 MB/s | 4.5 MB 00:00 2024-09-13T10:23:53.529 INFO:teuthology.orchestra.run.smithi173.stdout:(7/30): python3-ceph-argparse-18.2.4-843.g2c014 100 kB/s | 45 kB 00:00 2024-09-13T10:23:53.646 INFO:teuthology.orchestra.run.smithi173.stdout:(8/30): ceph-radosgw-18.2.4-843.g2c01458a.el9.x 4.2 MB/s | 7.7 MB 00:01 2024-09-13T10:23:53.671 INFO:teuthology.orchestra.run.smithi173.stdout:(9/30): python3-ceph-common-18.2.4-843.g2c01458 773 kB/s | 129 kB 00:00 2024-09-13T10:23:53.705 INFO:teuthology.orchestra.run.smithi173.stdout:(10/30): python3-cephfs-18.2.4-843.g2c01458a.el 923 kB/s | 162 kB 00:00 2024-09-13T10:23:53.780 INFO:teuthology.orchestra.run.smithi173.stdout:(11/30): python3-rados-18.2.4-843.g2c01458a.el9 2.3 MB/s | 321 kB 00:00 2024-09-13T10:23:53.880 INFO:teuthology.orchestra.run.smithi173.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 1.0 MB/s | 107 kB 00:00 2024-09-13T10:23:53.922 INFO:teuthology.orchestra.run.smithi138.stdout:(1/30): ceph-base-18.2.4-843.g2c01458a.el9.x86_ 9.8 MB/s | 5.1 MB 00:00 2024-09-13T10:23:53.939 INFO:teuthology.orchestra.run.smithi173.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-13T10:23:53.972 INFO:teuthology.orchestra.run.smithi138.stdout:(2/30): ceph-selinux-18.2.4-843.g2c01458a.el9.x 504 kB/s | 25 kB 00:00 2024-09-13T10:23:53.973 INFO:teuthology.orchestra.run.smithi173.stdout:(14/30): python3-rgw-18.2.4-843.g2c01458a.el9.x 372 kB/s | 99 kB 00:00 2024-09-13T10:23:53.998 INFO:teuthology.orchestra.run.smithi173.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-09-13T10:23:54.056 INFO:teuthology.orchestra.run.smithi173.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-13T10:23:54.064 INFO:teuthology.orchestra.run.smithi138.stdout:(3/30): libcephfs2-18.2.4-843.g2c01458a.el9.x86 7.6 MB/s | 712 kB 00:00 2024-09-13T10:23:54.098 INFO:teuthology.orchestra.run.smithi173.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-13T10:23:54.140 INFO:teuthology.orchestra.run.smithi173.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-13T10:23:54.156 INFO:teuthology.orchestra.run.smithi138.stdout:(4/30): libradosstriper1-18.2.4-843.g2c01458a.e 5.1 MB/s | 477 kB 00:00 2024-09-13T10:23:54.174 INFO:teuthology.orchestra.run.smithi173.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-13T10:23:54.238 INFO:teuthology.orchestra.run.smithi173.stdout:(20/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.1 MB/s | 308 kB 00:00 2024-09-13T10:23:54.274 INFO:teuthology.orchestra.run.smithi173.stdout:(21/30): python3-rbd-18.2.4-843.g2c01458a.el9.x 499 kB/s | 301 kB 00:00 2024-09-13T10:23:54.300 INFO:teuthology.orchestra.run.smithi173.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 434 kB/s | 26 kB 00:00 2024-09-13T10:23:54.334 INFO:teuthology.orchestra.run.smithi173.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-13T10:23:54.348 INFO:teuthology.orchestra.run.smithi138.stdout:(5/30): ceph-common-18.2.4-843.g2c01458a.el9.x8 19 MB/s | 18 MB 00:00 2024-09-13T10:23:54.367 INFO:teuthology.orchestra.run.smithi173.stdout:(24/30): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-13T10:23:54.409 INFO:teuthology.orchestra.run.smithi173.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-13T10:23:54.443 INFO:teuthology.orchestra.run.smithi173.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 839 kB 00:00 2024-09-13T10:23:54.448 INFO:teuthology.orchestra.run.smithi138.stdout:(6/30): ceph-radosgw-18.2.4-843.g2c01458a.el9.x 7.4 MB/s | 7.7 MB 00:01 2024-09-13T10:23:54.474 INFO:teuthology.orchestra.run.smithi138.stdout:(7/30): python3-ceph-argparse-18.2.4-843.g2c014 359 kB/s | 45 kB 00:00 2024-09-13T10:23:54.499 INFO:teuthology.orchestra.run.smithi138.stdout:(8/30): python3-ceph-common-18.2.4-843.g2c01458 2.5 MB/s | 129 kB 00:00 2024-09-13T10:23:54.525 INFO:teuthology.orchestra.run.smithi138.stdout:(9/30): python3-cephfs-18.2.4-843.g2c01458a.el9 3.1 MB/s | 162 kB 00:00 2024-09-13T10:23:54.526 INFO:teuthology.orchestra.run.smithi173.stdout:(27/30): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-13T10:23:54.550 INFO:teuthology.orchestra.run.smithi138.stdout:(10/30): python3-rados-18.2.4-843.g2c01458a.el9 6.2 MB/s | 321 kB 00:00 2024-09-13T10:23:54.575 INFO:teuthology.orchestra.run.smithi138.stdout:(11/30): python3-rbd-18.2.4-843.g2c01458a.el9.x 5.8 MB/s | 301 kB 00:00 2024-09-13T10:23:54.576 INFO:teuthology.orchestra.run.smithi173.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-13T10:23:54.601 INFO:teuthology.orchestra.run.smithi138.stdout:(12/30): python3-rgw-18.2.4-843.g2c01458a.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-13T10:23:54.676 INFO:teuthology.orchestra.run.smithi138.stdout:(13/30): librgw2-18.2.4-843.g2c01458a.el9.x86_6 8.6 MB/s | 4.5 MB 00:00 2024-09-13T10:23:54.776 INFO:teuthology.orchestra.run.smithi138.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2024-09-13T10:23:54.801 INFO:teuthology.orchestra.run.smithi138.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 473 kB/s | 107 kB 00:00 2024-09-13T10:23:54.844 INFO:teuthology.orchestra.run.smithi138.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 660 kB/s | 160 kB 00:00 2024-09-13T10:23:54.851 INFO:teuthology.orchestra.run.smithi173.stdout:(29/30): librbd1-18.2.4-843.g2c01458a.el9.x86_6 9.3 MB/s | 3.0 MB 00:00 2024-09-13T10:23:54.886 INFO:teuthology.orchestra.run.smithi138.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-13T10:23:54.893 INFO:teuthology.orchestra.run.smithi060.stdout:(1/30): ceph-base-18.2.4-843.g2c01458a.el9.x86_ 4.2 MB/s | 5.1 MB 00:01 2024-09-13T10:23:54.919 INFO:teuthology.orchestra.run.smithi138.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-13T10:23:54.943 INFO:teuthology.orchestra.run.smithi060.stdout:(2/30): ceph-selinux-18.2.4-843.g2c01458a.el9.x 507 kB/s | 25 kB 00:00 2024-09-13T10:23:54.960 INFO:teuthology.orchestra.run.smithi173.stdout:(30/30): librados2-18.2.4-843.g2c01458a.el9.x86 6.4 MB/s | 3.3 MB 00:00 2024-09-13T10:23:54.961 INFO:teuthology.orchestra.run.smithi138.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-13T10:23:54.966 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:23:54.966 INFO:teuthology.orchestra.run.smithi173.stdout:Total 16 MB/s | 53 MB 00:03 2024-09-13T10:23:54.995 INFO:teuthology.orchestra.run.smithi138.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.0 MB/s | 111 kB 00:00 2024-09-13T10:23:55.012 INFO:teuthology.orchestra.run.smithi138.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.7 MB/s | 26 kB 00:00 2024-09-13T10:23:55.027 INFO:teuthology.orchestra.run.smithi060.stdout:(3/30): libcephfs2-18.2.4-843.g2c01458a.el9.x86 8.3 MB/s | 712 kB 00:00 2024-09-13T10:23:55.029 INFO:teuthology.orchestra.run.smithi138.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-13T10:23:55.043 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:23:55.063 INFO:teuthology.orchestra.run.smithi138.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-13T10:23:55.079 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:23:55.079 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:23:55.096 INFO:teuthology.orchestra.run.smithi138.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-13T10:23:55.130 INFO:teuthology.orchestra.run.smithi138.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-13T10:23:55.161 INFO:teuthology.orchestra.run.smithi060.stdout:(4/30): ceph-radosgw-18.2.4-843.g2c01458a.el9.x 5.3 MB/s | 7.7 MB 00:01 2024-09-13T10:23:55.172 INFO:teuthology.orchestra.run.smithi138.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-13T10:23:55.186 INFO:teuthology.orchestra.run.smithi060.stdout:(5/30): libradosstriper1-18.2.4-843.g2c01458a.e 2.9 MB/s | 477 kB 00:00 2024-09-13T10:23:55.231 INFO:teuthology.orchestra.run.smithi138.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-13T10:23:55.236 INFO:teuthology.orchestra.run.smithi060.stdout:(6/30): python3-ceph-argparse-18.2.4-843.g2c014 904 kB/s | 45 kB 00:00 2024-09-13T10:23:55.303 INFO:teuthology.orchestra.run.smithi060.stdout:(7/30): python3-ceph-common-18.2.4-843.g2c01458 1.9 MB/s | 129 kB 00:00 2024-09-13T10:23:55.323 INFO:teuthology.orchestra.run.smithi138.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T10:23:55.379 INFO:teuthology.orchestra.run.smithi060.stdout:(8/30): python3-cephfs-18.2.4-843.g2c01458a.el9 2.1 MB/s | 162 kB 00:00 2024-09-13T10:23:55.440 INFO:teuthology.orchestra.run.smithi138.stdout:(29/30): librados2-18.2.4-843.g2c01458a.el9.x86 12 MB/s | 3.3 MB 00:00 2024-09-13T10:23:55.445 INFO:teuthology.orchestra.run.smithi060.stdout:(9/30): python3-rados-18.2.4-843.g2c01458a.el9. 4.7 MB/s | 321 kB 00:00 2024-09-13T10:23:55.521 INFO:teuthology.orchestra.run.smithi060.stdout:(10/30): librgw2-18.2.4-843.g2c01458a.el9.x86_6 12 MB/s | 4.5 MB 00:00 2024-09-13T10:23:55.532 INFO:teuthology.orchestra.run.smithi138.stdout:(30/30): librbd1-18.2.4-843.g2c01458a.el9.x86_6 10 MB/s | 3.0 MB 00:00 2024-09-13T10:23:55.537 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:23:55.537 INFO:teuthology.orchestra.run.smithi138.stdout:Total 22 MB/s | 53 MB 00:02 2024-09-13T10:23:55.554 INFO:teuthology.orchestra.run.smithi060.stdout:(11/30): python3-rbd-18.2.4-843.g2c01458a.el9.x 2.7 MB/s | 301 kB 00:00 2024-09-13T10:23:55.579 INFO:teuthology.orchestra.run.smithi060.stdout:(12/30): python3-rgw-18.2.4-843.g2c01458a.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-13T10:23:55.611 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:23:55.645 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:23:55.646 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:23:55.716 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:23:55.716 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:23:55.788 INFO:teuthology.orchestra.run.smithi060.stdout:(13/30): ceph-common-18.2.4-843.g2c01458a.el9.x 8.7 MB/s | 18 MB 00:02 2024-09-13T10:23:55.830 INFO:teuthology.orchestra.run.smithi060.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 388 kB/s | 107 kB 00:00 2024-09-13T10:23:55.855 INFO:teuthology.orchestra.run.smithi060.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 581 kB/s | 160 kB 00:00 2024-09-13T10:23:55.881 INFO:teuthology.orchestra.run.smithi060.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-09-13T10:23:55.924 INFO:teuthology.orchestra.run.smithi060.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-13T10:23:55.948 INFO:teuthology.orchestra.run.smithi060.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 632 kB/s | 42 kB 00:00 2024-09-13T10:23:55.973 INFO:teuthology.orchestra.run.smithi060.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-13T10:23:56.023 INFO:teuthology.orchestra.run.smithi060.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-13T10:23:56.049 INFO:teuthology.orchestra.run.smithi060.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T10:23:56.074 INFO:teuthology.orchestra.run.smithi060.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T10:23:56.099 INFO:teuthology.orchestra.run.smithi060.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-13T10:23:56.133 INFO:teuthology.orchestra.run.smithi060.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-13T10:23:56.167 INFO:teuthology.orchestra.run.smithi060.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-13T10:23:56.208 INFO:teuthology.orchestra.run.smithi060.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-13T10:23:56.267 INFO:teuthology.orchestra.run.smithi060.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-13T10:23:56.274 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:23:56.274 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:23:56.368 INFO:teuthology.orchestra.run.smithi060.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T10:23:56.501 INFO:teuthology.orchestra.run.smithi060.stdout:(29/30): librbd1-18.2.4-843.g2c01458a.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-13T10:23:56.618 INFO:teuthology.orchestra.run.smithi060.stdout:(30/30): librados2-18.2.4-843.g2c01458a.el9.x86 8.1 MB/s | 3.3 MB 00:00 2024-09-13T10:23:56.623 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:23:56.624 INFO:teuthology.orchestra.run.smithi060.stdout:Total 16 MB/s | 53 MB 00:03 2024-09-13T10:23:56.699 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:23:56.740 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:23:56.740 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:23:57.082 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:23:57.166 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T10:23:57.198 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T10:23:57.256 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T10:23:57.423 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:23:57.423 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:23:57.637 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T10:23:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:23:57.714 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:23:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T10:23:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T10:23:57.806 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:23:57.821 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T10:23:57.898 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 6/32 2024-09-13T10:23:57.924 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:23:57.987 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:23:58.024 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T10:23:58.067 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x 9/32 2024-09-13T10:23:58.134 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 10/32 2024-09-13T10:23:58.160 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:23:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T10:23:58.225 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:23:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:23:58.331 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T10:23:58.353 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:23:58.358 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T10:23:58.417 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T10:23:58.446 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 6/32 2024-09-13T10:23:58.449 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T10:23:58.476 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T10:23:58.489 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:23:58.520 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T10:23:58.552 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:23:58.554 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T10:23:58.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T10:23:58.633 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x 9/32 2024-09-13T10:23:58.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 10/32 2024-09-13T10:23:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:23:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:23:58.852 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T10:23:58.889 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T10:23:58.899 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 20/32 2024-09-13T10:23:58.924 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T10:23:58.927 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:23:58.969 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T10:23:58.998 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T10:23:59.000 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 20/32 2024-09-13T10:23:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T10:23:59.025 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T10:23:59.030 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 21/32 2024-09-13T10:23:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T10:23:59.061 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T10:23:59.064 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T10:23:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T10:23:59.094 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T10:23:59.393 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T10:23:59.441 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 20/32 2024-09-13T10:23:59.469 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T10:23:59.479 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T10:23:59.493 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 24/32 2024-09-13T10:23:59.540 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 20/32 2024-09-13T10:23:59.555 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:23:59.564 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 24/32 2024-09-13T10:23:59.579 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 21/32 2024-09-13T10:23:59.615 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T10:23:59.641 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 25/32 2024-09-13T10:23:59.648 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:23:59.679 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86 26/32 2024-09-13T10:23:59.734 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 6/32 2024-09-13T10:23:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:23:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:23:59.857 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T10:23:59.901 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x 9/32 2024-09-13T10:23:59.969 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 10/32 2024-09-13T10:23:59.992 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:24:00.020 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T10:24:00.053 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 24/32 2024-09-13T10:24:00.058 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:24:00.123 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 24/32 2024-09-13T10:24:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T10:24:00.191 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 25/32 2024-09-13T10:24:00.199 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T10:24:00.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86 26/32 2024-09-13T10:24:00.245 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T10:24:00.273 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T10:24:00.300 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T10:24:00.336 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T10:24:00.369 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T10:24:00.684 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T10:24:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 20/32 2024-09-13T10:24:00.823 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 20/32 2024-09-13T10:24:00.853 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 21/32 2024-09-13T10:24:00.890 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T10:24:01.316 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T10:24:01.342 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 24/32 2024-09-13T10:24:01.414 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 24/32 2024-09-13T10:24:01.493 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 25/32 2024-09-13T10:24:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86 26/32 2024-09-13T10:24:01.798 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:01.842 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:02.376 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:02.392 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:02.428 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:02.429 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 28/32 2024-09-13T10:24:02.529 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 28/32 2024-09-13T10:24:02.530 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T10:24:02.530 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T10:24:02.530 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:02.564 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:02.984 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 28/32 2024-09-13T10:24:03.366 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 28/32 2024-09-13T10:24:03.366 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T10:24:03.366 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T10:24:03.366 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:03.401 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 27/32 2024-09-13T10:24:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 28/32 2024-09-13T10:24:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 28/32 2024-09-13T10:24:04.486 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T10:24:04.486 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T10:24:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:12.661 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-09-13T10:24:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-09-13T10:24:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-09-13T10:24:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-09-13T10:24:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-09-13T10:24:12.662 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-09-13T10:24:12.662 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-09-13T10:24:12.662 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-09-13T10:24:13.209 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:13.430 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 30/32 2024-09-13T10:24:13.456 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 30/32 2024-09-13T10:24:13.456 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:13.456 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T10:24:13.456 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:24:13.456 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:24:13.457 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:13.458 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T10:24:13.549 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T10:24:13.550 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:13.951 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 30/32 2024-09-13T10:24:13.976 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 30/32 2024-09-13T10:24:13.976 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:13.976 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T10:24:13.976 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:24:13.977 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:24:13.977 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:13.978 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T10:24:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T10:24:14.053 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:14.709 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:14.709 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 1/32 2024-09-13T10:24:14.709 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 3/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 4/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 6/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x 8/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86 9/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 10/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 12/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 13/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T10:24:14.710 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T10:24:14.711 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T10:24:14.712 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T10:24:14.712 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T10:24:14.712 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T10:24:14.712 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:14.712 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T10:24:14.713 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 31/32 2024-09-13T10:24:15.191 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:15.191 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 1/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 3/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 4/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 6/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x 8/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86 9/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 10/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 12/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 13/32 2024-09-13T10:24:15.192 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T10:24:15.193 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T10:24:15.194 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T10:24:15.194 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:15.194 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T10:24:15.194 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 31/32 2024-09-13T10:24:15.265 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-09-13T10:24:15.266 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:15.691 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T10:24:15.692 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.693 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:15.694 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T10:24:15.694 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T10:24:15.694 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:15.694 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:16.037 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 30/32 2024-09-13T10:24:16.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-test 2024-09-13T10:24:16.064 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 30/32 2024-09-13T10:24:16.064 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:16.064 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T10:24:16.064 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:24:16.064 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:24:16.064 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:16.066 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T10:24:16.148 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T10:24:16.148 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.180 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T10:24:16.181 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:16.182 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:16.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-09-13T10:24:16.633 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:16.766 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:18.2.4-843.g2c01458a.el9 ceph 40 M 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:18.2.4-843.g2c01458a.el9 ceph 166 k 2024-09-13T10:24:16.767 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout:Install 4 Packages 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:16.768 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 40 M 2024-09-13T10:24:16.769 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 164 M 2024-09-13T10:24:16.769 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:17.006 INFO:teuthology.orchestra.run.smithi173.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-13T10:24:17.048 INFO:teuthology.orchestra.run.smithi173.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-13T10:24:17.073 INFO:teuthology.orchestra.run.smithi173.stdout:(3/4): libcephsqlite-18.2.4-843.g2c01458a.el9.x 649 kB/s | 166 kB 00:00 2024-09-13T10:24:17.116 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:17.255 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:17.255 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-13T10:24:17.255 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:17.255 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:18.2.4-843.g2c01458a.el9 ceph 40 M 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:18.2.4-843.g2c01458a.el9 ceph 166 k 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-09-13T10:24:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:17.257 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 40 M 2024-09-13T10:24:17.257 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 164 M 2024-09-13T10:24:17.257 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 1/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 3/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 4/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 5/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 6/32 2024-09-13T10:24:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 7/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x 8/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86 9/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 10/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 11/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 12/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 13/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T10:24:17.273 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T10:24:17.274 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T10:24:17.275 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T10:24:17.275 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:18.2.4-843.g2c01458a.el9.x86_64 29/32 2024-09-13T10:24:17.275 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T10:24:17.275 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 31/32 2024-09-13T10:24:17.623 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): libcephsqlite-18.2.4-843.g2c01458a.el9.x 733 kB/s | 166 kB 00:00 2024-09-13T10:24:18.182 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 388 kB/s | 304 kB 00:00 2024-09-13T10:24:18.299 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.300 INFO:teuthology.orchestra.run.smithi060.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T10:24:18.301 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:18.302 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:18.307 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-09-13T10:24:18.540 INFO:teuthology.orchestra.run.smithi173.stdout:(4/4): ceph-test-18.2.4-843.g2c01458a.el9.x86_6 23 MB/s | 40 MB 00:01 2024-09-13T10:24:18.541 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:18.542 INFO:teuthology.orchestra.run.smithi173.stdout:Total 23 MB/s | 40 MB 00:01 2024-09-13T10:24:18.570 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:18.590 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:18.590 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:18.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-test 2024-09-13T10:24:18.917 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:18.918 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:18.991 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): ceph-test-18.2.4-843.g2c01458a.el9.x86_6 25 MB/s | 40 MB 00:01 2024-09-13T10:24:18.993 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:18.993 INFO:teuthology.orchestra.run.smithi138.stdout:Total 23 MB/s | 40 MB 00:01 2024-09-13T10:24:19.021 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:19.042 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:19.042 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:19.252 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:29 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:19.364 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:19.365 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:19.395 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:19.399 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:19.399 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:18.2.4-843.g2c01458a.el9 ceph 40 M 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:18.2.4-843.g2c01458a.el9 ceph 166 k 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:19.400 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2024-09-13T10:24:19.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 40 M 2024-09-13T10:24:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 164 M 2024-09-13T10:24:19.401 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:19.488 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T10:24:19.538 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T10:24:19.583 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 3/4 2024-09-13T10:24:19.762 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): libcephsqlite-18.2.4-843.g2c01458a.el9.x 734 kB/s | 166 kB 00:00 2024-09-13T10:24:19.837 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:19.926 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T10:24:19.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T10:24:20.010 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 3/4 2024-09-13T10:24:20.329 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 384 kB/s | 304 kB 00:00 2024-09-13T10:24:20.388 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-13T10:24:20.805 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): ceph-test-18.2.4-843.g2c01458a.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-13T10:24:20.807 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:20.807 INFO:teuthology.orchestra.run.smithi060.stdout:Total 29 MB/s | 40 MB 00:01 2024-09-13T10:24:20.835 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:20.857 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:20.857 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:21.194 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:21.195 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:21.684 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:21.786 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T10:24:21.830 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T10:24:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 3/4 2024-09-13T10:24:23.315 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 3/4 2024-09-13T10:24:23.390 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 4/4 2024-09-13T10:24:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 3/4 2024-09-13T10:24:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 4/4 2024-09-13T10:24:25.519 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 4/4 2024-09-13T10:24:25.519 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 1/4 2024-09-13T10:24:25.519 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 2/4 2024-09-13T10:24:25.519 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T10:24:25.775 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 3/4 2024-09-13T10:24:25.844 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T10:24:25.844 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:25.844 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:25.844 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:25.844 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:25.845 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T10:24:25.845 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T10:24:25.845 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:25.845 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 4/4 2024-09-13T10:24:25.994 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 4/4 2024-09-13T10:24:25.995 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 1/4 2024-09-13T10:24:25.995 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 2/4 2024-09-13T10:24:25.995 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T10:24:26.151 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T10:24:26.310 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T10:24:26.311 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:26.311 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:26.634 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-09-13T10:24:26.719 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:37 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:26.854 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:26.856 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:18.2.4-843.g2c01458a.el9 ceph 6.5 k 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:18.2.4-843.g2c01458a.el9 ceph 2.1 M 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:18.2.4-843.g2c01458a.el9 ceph 1.5 M 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:18.2.4-843.g2c01458a.el9 ceph-noarch 246 k 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:18.2.4-843.g2c01458a.el9 ceph 4.7 M 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 17 M 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T10:24:26.857 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T10:24:26.858 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T10:24:26.859 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout:Install 33 Packages 2024-09-13T10:24:26.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:26.861 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 29 M 2024-09-13T10:24:26.862 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 104 M 2024-09-13T10:24:26.862 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:27.200 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:37 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:27.340 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:18.2.4-843.g2c01458a.el9 ceph 6.5 k 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:18.2.4-843.g2c01458a.el9 ceph 2.1 M 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:18.2.4-843.g2c01458a.el9 ceph 1.5 M 2024-09-13T10:24:27.343 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:18.2.4-843.g2c01458a.el9 ceph-noarch 246 k 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:18.2.4-843.g2c01458a.el9 ceph 4.7 M 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 17 M 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T10:24:27.344 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T10:24:27.345 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:27.346 INFO:teuthology.orchestra.run.smithi138.stdout:Install 33 Packages 2024-09-13T10:24:27.347 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:27.347 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 29 M 2024-09-13T10:24:27.347 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 104 M 2024-09-13T10:24:27.348 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:27.807 INFO:teuthology.orchestra.run.smithi173.stdout:(1/33): ceph-18.2.4-843.g2c01458a.el9.x86_64.rp 22 kB/s | 6.5 kB 00:00 2024-09-13T10:24:27.822 INFO:teuthology.orchestra.run.smithi138.stdout:(1/33): ceph-18.2.4-843.g2c01458a.el9.x86_64.rp 23 kB/s | 6.5 kB 00:00 2024-09-13T10:24:27.858 INFO:teuthology.orchestra.run.smithi173.stdout:(2/33): ceph-mds-18.2.4-843.g2c01458a.el9.x86_6 6.0 MB/s | 2.1 MB 00:00 2024-09-13T10:24:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 4/4 2024-09-13T10:24:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 1/4 2024-09-13T10:24:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 2/4 2024-09-13T10:24:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T10:24:27.924 INFO:teuthology.orchestra.run.smithi173.stdout:(3/33): ceph-mgr-18.2.4-843.g2c01458a.el9.x86_6 3.5 MB/s | 1.5 MB 00:00 2024-09-13T10:24:28.030 INFO:teuthology.orchestra.run.smithi138.stdout:(2/33): ceph-mds-18.2.4-843.g2c01458a.el9.x86_6 4.3 MB/s | 2.1 MB 00:00 2024-09-13T10:24:28.070 INFO:teuthology.orchestra.run.smithi173.stdout:(4/33): ceph-mgr-modules-core-18.2.4-843.g2c014 1.7 MB/s | 246 kB 00:00 2024-09-13T10:24:28.072 INFO:teuthology.orchestra.run.smithi138.stdout:(3/33): ceph-mgr-18.2.4-843.g2c01458a.el9.x86_6 2.7 MB/s | 1.5 MB 00:00 2024-09-13T10:24:28.248 INFO:teuthology.orchestra.run.smithi173.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-13T10:24:28.362 INFO:teuthology.orchestra.run.smithi173.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 10 MB/s | 1.2 MB 00:00 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:28.369 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:28.422 INFO:teuthology.orchestra.run.smithi138.stdout:(4/33): ceph-mgr-modules-core-18.2.4-843.g2c014 702 kB/s | 246 kB 00:00 2024-09-13T10:24:28.529 INFO:teuthology.orchestra.run.smithi173.stdout:(7/33): ceph-osd-18.2.4-843.g2c01458a.el9.x86_6 25 MB/s | 17 MB 00:00 2024-09-13T10:24:28.554 INFO:teuthology.orchestra.run.smithi173.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-09-13T10:24:28.589 INFO:teuthology.orchestra.run.smithi138.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-13T10:24:28.604 INFO:teuthology.orchestra.run.smithi173.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.5 MB/s | 125 kB 00:00 2024-09-13T10:24:28.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph 2024-09-13T10:24:28.680 INFO:teuthology.orchestra.run.smithi173.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-13T10:24:28.705 INFO:teuthology.orchestra.run.smithi173.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 770 kB/s | 135 kB 00:00 2024-09-13T10:24:28.706 INFO:teuthology.orchestra.run.smithi138.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 11 MB/s | 1.2 MB 00:00 2024-09-13T10:24:28.772 INFO:teuthology.orchestra.run.smithi173.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 521 kB/s | 35 kB 00:00 2024-09-13T10:24:28.814 INFO:teuthology.orchestra.run.smithi173.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-13T10:24:28.839 INFO:teuthology.orchestra.run.smithi173.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 762 kB/s | 51 kB 00:00 2024-09-13T10:24:28.865 INFO:teuthology.orchestra.run.smithi173.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-13T10:24:28.869 INFO:teuthology.orchestra.run.smithi138.stdout:(7/33): ceph-osd-18.2.4-843.g2c01458a.el9.x86_6 20 MB/s | 17 MB 00:00 2024-09-13T10:24:28.890 INFO:teuthology.orchestra.run.smithi138.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 579 kB/s | 106 kB 00:00 2024-09-13T10:24:28.941 INFO:teuthology.orchestra.run.smithi138.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-13T10:24:28.957 INFO:teuthology.orchestra.run.smithi173.stdout:(16/33): ceph-mon-18.2.4-843.g2c01458a.el9.x86_ 4.1 MB/s | 4.7 MB 00:01 2024-09-13T10:24:28.982 INFO:teuthology.orchestra.run.smithi173.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 305 kB/s | 43 kB 00:00 2024-09-13T10:24:28.999 INFO:teuthology.orchestra.run.smithi138.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-13T10:24:29.016 INFO:teuthology.orchestra.run.smithi173.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-13T10:24:29.025 INFO:teuthology.orchestra.run.smithi138.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 848 kB/s | 135 kB 00:00 2024-09-13T10:24:29.041 INFO:teuthology.orchestra.run.smithi173.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 209 kB/s | 11 kB 00:00 2024-09-13T10:24:29.066 INFO:teuthology.orchestra.run.smithi173.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 366 kB/s | 18 kB 00:00 2024-09-13T10:24:29.091 INFO:teuthology.orchestra.run.smithi173.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-09-13T10:24:29.117 INFO:teuthology.orchestra.run.smithi138.stdout:(12/33): ceph-mon-18.2.4-843.g2c01458a.el9.x86_ 3.6 MB/s | 4.7 MB 00:01 2024-09-13T10:24:29.118 INFO:teuthology.orchestra.run.smithi173.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-13T10:24:29.142 INFO:teuthology.orchestra.run.smithi173.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-13T10:24:29.167 INFO:teuthology.orchestra.run.smithi173.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-13T10:24:29.193 INFO:teuthology.orchestra.run.smithi173.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T10:24:29.218 INFO:teuthology.orchestra.run.smithi173.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 655 kB/s | 16 kB 00:00 2024-09-13T10:24:29.233 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:39 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:29.243 INFO:teuthology.orchestra.run.smithi173.stdout:(27/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-13T10:24:29.269 INFO:teuthology.orchestra.run.smithi173.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-13T10:24:29.294 INFO:teuthology.orchestra.run.smithi173.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-13T10:24:29.317 INFO:teuthology.orchestra.run.smithi138.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 119 kB/s | 35 kB 00:00 2024-09-13T10:24:29.319 INFO:teuthology.orchestra.run.smithi173.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-13T10:24:29.344 INFO:teuthology.orchestra.run.smithi173.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-09-13T10:24:29.369 INFO:teuthology.orchestra.run.smithi173.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-13T10:24:29.377 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:29.379 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:29.379 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-13T10:24:29.379 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:29.379 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:29.379 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:18.2.4-843.g2c01458a.el9 ceph 6.5 k 2024-09-13T10:24:29.379 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:18.2.4-843.g2c01458a.el9 ceph 2.1 M 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:18.2.4-843.g2c01458a.el9 ceph 1.5 M 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:18.2.4-843.g2c01458a.el9 ceph-noarch 246 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:18.2.4-843.g2c01458a.el9 ceph 4.7 M 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 17 M 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T10:24:29.380 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T10:24:29.381 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout:Install 33 Packages 2024-09-13T10:24:29.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:29.383 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 29 M 2024-09-13T10:24:29.383 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 104 M 2024-09-13T10:24:29.383 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:29.395 INFO:teuthology.orchestra.run.smithi173.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-13T10:24:29.399 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:29.400 INFO:teuthology.orchestra.run.smithi173.stdout:Total 11 MB/s | 29 MB 00:02 2024-09-13T10:24:29.493 INFO:teuthology.orchestra.run.smithi138.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 136 kB/s | 51 kB 00:00 2024-09-13T10:24:29.518 INFO:teuthology.orchestra.run.smithi138.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-09-13T10:24:29.523 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:29.560 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:29.560 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:29.560 INFO:teuthology.orchestra.run.smithi138.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-13T10:24:29.602 INFO:teuthology.orchestra.run.smithi138.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 8.8 MB/s | 358 kB 00:00 2024-09-13T10:24:29.627 INFO:teuthology.orchestra.run.smithi138.stdout:(18/33): python3-mako-1.1.4-6.el9.noarch.rpm 274 kB/s | 172 kB 00:00 2024-09-13T10:24:29.653 INFO:teuthology.orchestra.run.smithi138.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-09-13T10:24:29.686 INFO:teuthology.orchestra.run.smithi138.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-13T10:24:29.712 INFO:teuthology.orchestra.run.smithi138.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 452 kB/s | 23 kB 00:00 2024-09-13T10:24:29.737 INFO:teuthology.orchestra.run.smithi138.stdout:(22/33): python3-jaraco-classes-3.2.1-5.el9.noa 162 kB/s | 18 kB 00:00 2024-09-13T10:24:29.762 INFO:teuthology.orchestra.run.smithi138.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 263 kB/s | 19 kB 00:00 2024-09-13T10:24:29.788 INFO:teuthology.orchestra.run.smithi138.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-13T10:24:29.813 INFO:teuthology.orchestra.run.smithi138.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-13T10:24:29.838 INFO:teuthology.orchestra.run.smithi138.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-13T10:24:29.864 INFO:teuthology.orchestra.run.smithi138.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-13T10:24:29.889 INFO:teuthology.orchestra.run.smithi138.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-13T10:24:29.914 INFO:teuthology.orchestra.run.smithi138.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-13T10:24:29.939 INFO:teuthology.orchestra.run.smithi138.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-13T10:24:29.965 INFO:teuthology.orchestra.run.smithi138.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 803 kB/s | 20 kB 00:00 2024-09-13T10:24:29.965 INFO:teuthology.orchestra.run.smithi060.stdout:(1/33): ceph-18.2.4-843.g2c01458a.el9.x86_64.rp 27 kB/s | 6.5 kB 00:00 2024-09-13T10:24:29.966 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:29.967 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:29.990 INFO:teuthology.orchestra.run.smithi138.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-13T10:24:30.015 INFO:teuthology.orchestra.run.smithi138.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-13T10:24:30.020 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:30.021 INFO:teuthology.orchestra.run.smithi138.stdout:Total 11 MB/s | 29 MB 00:02 2024-09-13T10:24:30.144 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:30.149 INFO:teuthology.orchestra.run.smithi060.stdout:(2/33): ceph-mgr-18.2.4-843.g2c01458a.el9.x86_6 3.4 MB/s | 1.5 MB 00:00 2024-09-13T10:24:30.182 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:30.182 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:30.232 INFO:teuthology.orchestra.run.smithi060.stdout:(3/33): ceph-mds-18.2.4-843.g2c01458a.el9.x86_6 4.1 MB/s | 2.1 MB 00:00 2024-09-13T10:24:30.283 INFO:teuthology.orchestra.run.smithi060.stdout:(4/33): ceph-mgr-modules-core-18.2.4-843.g2c014 4.8 MB/s | 246 kB 00:00 2024-09-13T10:24:30.433 INFO:teuthology.orchestra.run.smithi060.stdout:(5/33): ceph-mon-18.2.4-843.g2c01458a.el9.x86_6 10 MB/s | 4.7 MB 00:00 2024-09-13T10:24:30.492 INFO:teuthology.orchestra.run.smithi060.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-13T10:24:30.542 INFO:teuthology.orchestra.run.smithi060.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-13T10:24:30.588 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:30.588 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:30.601 INFO:teuthology.orchestra.run.smithi060.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-13T10:24:30.659 INFO:teuthology.orchestra.run.smithi060.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-09-13T10:24:30.860 INFO:teuthology.orchestra.run.smithi060.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.0 MB/s | 215 kB 00:00 2024-09-13T10:24:30.902 INFO:teuthology.orchestra.run.smithi060.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-13T10:24:31.127 INFO:teuthology.orchestra.run.smithi060.stdout:(12/33): ceph-osd-18.2.4-843.g2c01458a.el9.x86_ 17 MB/s | 17 MB 00:00 2024-09-13T10:24:31.219 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:31.290 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T10:24:31.322 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T10:24:31.327 INFO:teuthology.orchestra.run.smithi060.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 82 kB/s | 35 kB 00:00 2024-09-13T10:24:31.363 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T10:24:31.471 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T10:24:31.494 INFO:teuthology.orchestra.run.smithi060.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 139 kB/s | 51 kB 00:00 2024-09-13T10:24:31.528 INFO:teuthology.orchestra.run.smithi060.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-09-13T10:24:31.550 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T10:24:31.561 INFO:teuthology.orchestra.run.smithi060.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 648 kB/s | 43 kB 00:00 2024-09-13T10:24:31.587 INFO:teuthology.orchestra.run.smithi060.stdout:(17/33): python3-mako-1.1.4-6.el9.noarch.rpm 237 kB/s | 172 kB 00:00 2024-09-13T10:24:31.591 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T10:24:31.624 INFO:teuthology.orchestra.run.smithi060.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-13T10:24:31.626 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T10:24:31.649 INFO:teuthology.orchestra.run.smithi060.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 173 kB/s | 11 kB 00:00 2024-09-13T10:24:31.672 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T10:24:31.674 INFO:teuthology.orchestra.run.smithi060.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 367 kB/s | 18 kB 00:00 2024-09-13T10:24:31.699 INFO:teuthology.orchestra.run.smithi060.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 943 kB/s | 23 kB 00:00 2024-09-13T10:24:31.707 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T10:24:31.725 INFO:teuthology.orchestra.run.smithi060.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 391 kB/s | 19 kB 00:00 2024-09-13T10:24:31.740 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T10:24:31.751 INFO:teuthology.orchestra.run.smithi060.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-09-13T10:24:31.775 INFO:teuthology.orchestra.run.smithi060.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-13T10:24:31.781 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T10:24:31.809 INFO:teuthology.orchestra.run.smithi060.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 552 kB/s | 46 kB 00:00 2024-09-13T10:24:31.824 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T10:24:31.834 INFO:teuthology.orchestra.run.smithi060.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 940 kB/s | 79 kB 00:00 2024-09-13T10:24:31.841 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:31.859 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T10:24:31.859 INFO:teuthology.orchestra.run.smithi060.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-13T10:24:31.885 INFO:teuthology.orchestra.run.smithi060.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-13T10:24:31.906 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T10:24:31.911 INFO:teuthology.orchestra.run.smithi060.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-13T10:24:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T10:24:31.935 INFO:teuthology.orchestra.run.smithi060.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-13T10:24:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T10:24:31.961 INFO:teuthology.orchestra.run.smithi060.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-13T10:24:31.976 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T10:24:31.986 INFO:teuthology.orchestra.run.smithi060.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-13T10:24:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T10:24:32.011 INFO:teuthology.orchestra.run.smithi060.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-13T10:24:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:Total 11 MB/s | 29 MB 00:02 2024-09-13T10:24:32.071 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T10:24:32.141 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:32.148 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T10:24:32.164 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T10:24:32.182 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:32.182 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T10:24:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T10:24:32.246 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T10:24:32.262 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T10:24:32.305 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T10:24:32.309 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T10:24:32.338 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T10:24:32.347 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T10:24:32.393 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T10:24:32.396 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T10:24:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T10:24:32.451 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T10:24:32.482 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T10:24:32.532 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T10:24:32.593 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:32.593 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:32.629 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T10:24:32.636 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T10:24:32.685 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T10:24:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T10:24:32.790 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T10:24:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T10:24:32.876 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T10:24:32.899 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T10:24:32.929 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T10:24:32.967 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T10:24:33.016 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T10:24:33.025 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T10:24:33.081 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T10:24:33.176 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.n 28/33 2024-09-13T10:24:33.202 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 29/33 2024-09-13T10:24:33.229 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 29/33 2024-09-13T10:24:33.229 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:33.229 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T10:24:33.230 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T10:24:33.230 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T10:24:33.230 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:33.267 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T10:24:33.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T10:24:33.409 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T10:24:33.497 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T10:24:33.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T10:24:33.655 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T10:24:33.814 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.n 28/33 2024-09-13T10:24:33.841 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 29/33 2024-09-13T10:24:33.867 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 29/33 2024-09-13T10:24:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T10:24:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T10:24:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T10:24:33.868 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T10:24:34.180 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T10:24:34.213 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T10:24:34.308 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T10:24:34.510 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T10:24:34.558 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T10:24:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T10:24:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T10:24:34.637 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 30/33 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 30/33 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T10:24:34.707 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T10:24:34.747 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T10:24:34.791 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T10:24:34.825 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T10:24:34.868 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T10:24:34.948 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T10:24:35.108 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T10:24:35.112 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 31/33 2024-09-13T10:24:35.137 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 31/33 2024-09-13T10:24:35.137 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:35.137 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T10:24:35.137 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T10:24:35.137 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T10:24:35.137 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:35.190 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T10:24:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T10:24:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T10:24:35.292 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 30/33 2024-09-13T10:24:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 30/33 2024-09-13T10:24:35.320 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:35.320 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T10:24:35.320 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T10:24:35.320 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T10:24:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:35.347 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 32/33 2024-09-13T10:24:35.364 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T10:24:35.371 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 32/33 2024-09-13T10:24:35.371 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:35.371 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T10:24:35.371 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T10:24:35.371 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T10:24:35.371 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:35.428 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T10:24:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:18.2.4-843.g2c01458a.el9.x86_64 33/33 2024-09-13T10:24:35.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T10:24:35.685 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T10:24:35.765 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T10:24:35.766 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 31/33 2024-09-13T10:24:35.790 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 31/33 2024-09-13T10:24:35.790 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:35.790 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T10:24:35.790 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T10:24:35.790 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T10:24:35.790 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:35.854 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T10:24:35.903 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T10:24:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 32/33 2024-09-13T10:24:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T10:24:36.032 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 32/33 2024-09-13T10:24:36.032 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:36.032 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T10:24:36.032 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T10:24:36.032 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T10:24:36.032 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:36.098 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:18.2.4-843.g2c01458a.el9.x86_64 33/33 2024-09-13T10:24:36.198 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.n 28/33 2024-09-13T10:24:36.227 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 29/33 2024-09-13T10:24:36.254 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 29/33 2024-09-13T10:24:36.254 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:36.254 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T10:24:36.254 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T10:24:36.254 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T10:24:36.254 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-2:18.2.4-843.g2c01458a.el9.x86_64 33/33 2024-09-13T10:24:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:18.2.4-843.g2c01458a.el9.x86_64 1/33 2024-09-13T10:24:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 2/33 2024-09-13T10:24:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 3/33 2024-09-13T10:24:37.192 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 4/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 5/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.n 6/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T10:24:37.193 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T10:24:37.194 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T10:24:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T10:24:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T10:24:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T10:24:37.195 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T10:24:37.762 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 30/33 2024-09-13T10:24:37.793 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 30/33 2024-09-13T10:24:37.793 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:37.793 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T10:24:37.793 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T10:24:37.793 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T10:24:37.793 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:18.2.4-843.g2c01458a.el9.x86_64 33/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:18.2.4-843.g2c01458a.el9.x86_64 1/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 2/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 3/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 4/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 5/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.n 6/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T10:24:37.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T10:24:37.843 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T10:24:37.844 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T10:24:37.844 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T10:24:37.845 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T10:24:37.845 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T10:24:37.845 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T10:24:37.845 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T10:24:37.845 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T10:24:37.845 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T10:24:38.107 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.noarch 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T10:24:38.108 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T10:24:38.109 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:38.110 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:38.254 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 31/33 2024-09-13T10:24:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 31/33 2024-09-13T10:24:38.280 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:38.280 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T10:24:38.280 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T10:24:38.280 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T10:24:38.280 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:38.491 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-base 2024-09-13T10:24:38.505 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 32/33 2024-09-13T10:24:38.530 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 32/33 2024-09-13T10:24:38.530 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:38.530 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T10:24:38.531 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T10:24:38.531 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T10:24:38.531 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:38.613 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:18.2.4-843.g2c01458a.el9.x86_64 33/33 2024-09-13T10:24:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T10:24:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:38.838 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.noarch 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T10:24:38.839 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T10:24:38.840 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:38.841 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:39.063 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:39.153 INFO:teuthology.orchestra.run.smithi173.stdout:Package ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:39.195 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:39.196 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:39.196 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:39.241 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-09-13T10:24:39.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install cephadm 2024-09-13T10:24:39.814 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:39.835 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:39.903 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:39.946 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:39.947 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:39.947 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:39.968 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:39.969 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:39.969 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-13T10:24:39.969 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:39.969 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:39.969 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:18.2.4-843.g2c01458a.el9 ceph-noarch 226 k 2024-09-13T10:24:39.969 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 226 k 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 217 k 2024-09-13T10:24:39.970 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:40.017 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-09-13T10:24:40.239 INFO:teuthology.orchestra.run.smithi173.stdout:cephadm-18.2.4-843.g2c01458a.el9.noarch.rpm 843 kB/s | 226 kB 00:00 2024-09-13T10:24:40.239 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:40.239 INFO:teuthology.orchestra.run.smithi173.stdout:Total 837 kB/s | 226 kB 00:00 2024-09-13T10:24:40.240 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:40.245 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:40.245 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:40.248 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:40.248 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:40.322 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-2:18.2.4-843.g2c01458a.el9.x86_64 33/33 2024-09-13T10:24:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:18.2.4-843.g2c01458a.el9.x86_64 1/33 2024-09-13T10:24:40.362 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 2/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 3/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 4/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 5/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.n 6/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T10:24:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T10:24:40.364 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T10:24:40.365 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T10:24:40.581 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:40.713 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:40.713 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:40.713 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-13T10:24:40.713 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:40.713 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:18.2.4-843.g2c01458a.el9 ceph-noarch 226 k 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:40.714 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 226 k 2024-09-13T10:24:40.715 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 217 k 2024-09-13T10:24:40.715 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:40.734 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:40.810 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:40.928 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-18.2.4-843.g2c01458a.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-13T10:24:40.928 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:40.929 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-13T10:24:40.929 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:40.934 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:40.934 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:40.940 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:40.940 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:41.167 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:41.385 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:18.2.4-843.g2c01458a.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T10:24:41.386 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T10:24:41.387 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T10:24:41.388 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T10:24:41.388 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T10:24:41.388 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:41.388 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:41.436 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 2024-09-13T10:24:41.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:41.436 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:41.485 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:41.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:41.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T10:24:41.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-base 2024-09-13T10:24:41.913 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:42.150 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:42.151 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:42.151 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:42.151 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 2024-09-13T10:24:42.151 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:42.151 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:42.247 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:42.358 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T10:24:42.369 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:42.383 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:42.383 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:42.383 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-13T10:24:42.383 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-843.g2c01458a.el9 ceph 143 k 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 143 k 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 395 k 2024-09-13T10:24:42.384 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:42.462 INFO:teuthology.orchestra.run.smithi060.stdout:Package ceph-base-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:42.506 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:42.507 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:42.507 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:42.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install cephadm 2024-09-13T10:24:42.587 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-immutable-object-cache-18.2.4-843.g2c01458 705 kB/s | 143 kB 00:00 2024-09-13T10:24:42.588 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:42.588 INFO:teuthology.orchestra.run.smithi173.stdout:Total 698 kB/s | 143 kB 00:00 2024-09-13T10:24:42.588 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:42.596 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:42.596 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:42.654 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:42.654 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:42.793 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:42.880 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:42.905 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:42.905 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:42.905 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T10:24:42.905 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:42.936 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:43.069 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:43.070 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:43.070 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-13T10:24:43.070 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-843.g2c01458a.el9 ceph 143 k 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 143 k 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 395 k 2024-09-13T10:24:43.071 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:43.163 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:43.271 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-18.2.4-843.g2c01458 718 kB/s | 143 kB 00:00 2024-09-13T10:24:43.271 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:43.272 INFO:teuthology.orchestra.run.smithi138.stdout:Total 711 kB/s | 143 kB 00:00 2024-09-13T10:24:43.272 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:43.279 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:43.279 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:18.2.4-843.g2c01458a.el9 ceph-noarch 226 k 2024-09-13T10:24:43.300 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 226 k 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 217 k 2024-09-13T10:24:43.301 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:43.335 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:43.336 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:43.491 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:43.523 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm-18.2.4-843.g2c01458a.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-13T10:24:43.524 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:43.524 INFO:teuthology.orchestra.run.smithi060.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-13T10:24:43.524 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:43.529 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:43.529 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:43.532 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:43.533 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:43.589 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:43.589 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:43.589 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T10:24:43.589 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:43.803 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:43.803 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:43.804 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:43.804 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:43.804 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:43.804 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:44.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-mgr 2024-09-13T10:24:44.063 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:44.154 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:44.472 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:44.472 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:44.472 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:44.472 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:44.472 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:44.472 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:44.521 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:44.604 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:44.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-09-13T10:24:44.695 INFO:teuthology.orchestra.run.smithi173.stdout:Package ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:44.737 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:44.737 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:44.738 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:18.2.4-843.g2c01458a.el9.noarch 1/1 2024-09-13T10:24:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:44.792 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:18.2.4-843.g2c01458a.el9.noarch 2024-09-13T10:24:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:44.793 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:44.808 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-fuse 2024-09-13T10:24:45.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T10:24:45.273 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:45.366 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:45.372 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:45.407 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:45.409 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:45.409 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:45.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-09-13T10:24:45.503 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 852 k 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:45.504 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:45.505 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:45.505 INFO:teuthology.orchestra.run.smithi173.stdout:Install 2 Packages 2024-09-13T10:24:45.505 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:45.505 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 932 k 2024-09-13T10:24:45.505 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 2.7 M 2024-09-13T10:24:45.505 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:45.611 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:45.751 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-843.g2c01458a.el9 ceph 143 k 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:45.752 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 143 k 2024-09-13T10:24:45.753 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 395 k 2024-09-13T10:24:45.753 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:45.899 INFO:teuthology.orchestra.run.smithi173.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 557 kB/s | 80 kB 00:00 2024-09-13T10:24:45.963 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-immutable-object-cache-18.2.4-843.g2c01458 679 kB/s | 143 kB 00:00 2024-09-13T10:24:45.963 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:45.963 INFO:teuthology.orchestra.run.smithi060.stdout:Total 675 kB/s | 143 kB 00:00 2024-09-13T10:24:45.964 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:45.972 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:46.033 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:46.033 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:46.052 INFO:teuthology.orchestra.run.smithi173.stdout:(2/2): ceph-fuse-18.2.4-843.g2c01458a.el9.x86_6 2.8 MB/s | 852 kB 00:00 2024-09-13T10:24:46.052 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:46.052 INFO:teuthology.orchestra.run.smithi173.stdout:Total 1.7 MB/s | 932 kB 00:00 2024-09-13T10:24:46.052 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:46.063 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:46.085 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:46.085 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:46.158 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:46.159 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 852 k 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-13T10:24:46.183 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 932 k 2024-09-13T10:24:46.184 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.7 M 2024-09-13T10:24:46.185 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:46.247 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:46.274 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:46.274 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:24:46.274 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T10:24:46.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:46.345 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:46.490 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T10:24:46.564 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2/2 2024-09-13T10:24:46.727 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): ceph-fuse-18.2.4-843.g2c01458a.el9.x86_6 3.1 MB/s | 852 kB 00:00 2024-09-13T10:24:46.927 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 171 kB/s | 80 kB 00:00 2024-09-13T10:24:46.927 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:46.928 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.2 MB/s | 932 kB 00:00 2024-09-13T10:24:46.940 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:46.963 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:46.963 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:47.037 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:47.038 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:47.218 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.e 1/1 2024-09-13T10:24:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:47.234 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:47.234 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2/2 2024-09-13T10:24:47.327 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/2 2024-09-13T10:24:47.360 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T10:24:47.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2/2 2024-09-13T10:24:47.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr 2024-09-13T10:24:47.594 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T10:24:47.594 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:47.594 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:47.594 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T10:24:47.594 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:47.594 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:47.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librados-devel 2024-09-13T10:24:48.054 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:58 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:48.136 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2/2 2024-09-13T10:24:48.136 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/2 2024-09-13T10:24:48.149 INFO:teuthology.orchestra.run.smithi060.stdout:Package ceph-mgr-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:48.192 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:48.193 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:48.194 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:48.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-fuse 2024-09-13T10:24:48.405 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T10:24:48.405 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:48.405 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:48.405 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T10:24:48.406 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:48.406 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:48.463 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:48.600 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:48.600 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:48.600 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:18.2.4-843.g2c01458a.el9 ceph 127 k 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 127 k 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 456 k 2024-09-13T10:24:48.601 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:48.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-09-13T10:24:48.821 INFO:teuthology.orchestra.run.smithi173.stdout:librados-devel-18.2.4-843.g2c01458a.el9.x86_64. 577 kB/s | 127 kB 00:00 2024-09-13T10:24:48.822 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:48.822 INFO:teuthology.orchestra.run.smithi173.stdout:Total 573 kB/s | 127 kB 00:00 2024-09-13T10:24:48.822 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:48.827 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:48.828 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:48.852 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:58 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:48.884 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:48.885 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-13T10:24:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 852 k 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout:Install 2 Packages 2024-09-13T10:24:48.991 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 932 k 2024-09-13T10:24:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 2.7 M 2024-09-13T10:24:48.992 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:49.032 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:49.133 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:49.201 INFO:teuthology.orchestra.run.smithi060.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 754 kB/s | 80 kB 00:00 2024-09-13T10:24:49.216 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:49.352 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:49.354 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-13T10:24:49.354 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:49.354 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:18.2.4-843.g2c01458a.el9 ceph 127 k 2024-09-13T10:24:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:49.355 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:49.355 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:49.355 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:24:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:49.355 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 127 k 2024-09-13T10:24:49.356 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 456 k 2024-09-13T10:24:49.356 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:49.401 INFO:teuthology.orchestra.run.smithi060.stdout:(2/2): ceph-fuse-18.2.4-843.g2c01458a.el9.x86_6 2.7 MB/s | 852 kB 00:00 2024-09-13T10:24:49.402 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:49.402 INFO:teuthology.orchestra.run.smithi060.stdout:Total 2.2 MB/s | 932 kB 00:00 2024-09-13T10:24:49.414 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:49.437 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:49.437 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:49.516 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:49.516 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:49.538 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:49.560 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-18.2.4-843.g2c01458a.el9.x86_64. 614 kB/s | 127 kB 00:00 2024-09-13T10:24:49.561 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:49.561 INFO:teuthology.orchestra.run.smithi138.stdout:Total 610 kB/s | 127 kB 00:00 2024-09-13T10:24:49.562 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:49.567 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:49.567 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:49.625 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:49.625 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:49.698 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:49.775 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:49.801 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:49.801 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:49.801 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:49.801 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:49.801 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:49.801 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:49.845 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T10:24:49.881 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:49.918 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2/2 2024-09-13T10:24:50.036 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install libcephfs2 2024-09-13T10:24:50.276 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:50.530 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:50.530 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:50.600 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:50.633 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2/2 2024-09-13T10:24:50.633 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/2 2024-09-13T10:24:50.691 INFO:teuthology.orchestra.run.smithi173.stdout:Package libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:50.733 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:50.734 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:50.734 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:50.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-09-13T10:24:50.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install libcephfs-devel 2024-09-13T10:24:50.918 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T10:24:50.918 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:50.918 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:50.918 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T10:24:50.918 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:50.918 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:51.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librados-devel 2024-09-13T10:24:51.379 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:02 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:51.477 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:51.510 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:18.2.4-843.g2c01458a.el9 ceph 31 k 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:24:51.511 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:51.512 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 31 k 2024-09-13T10:24:51.512 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 139 k 2024-09-13T10:24:51.512 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:51.520 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:51.521 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:51.521 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:51.600 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-09-13T10:24:51.668 INFO:teuthology.orchestra.run.smithi173.stdout:libcephfs-devel-18.2.4-843.g2c01458a.el9.x86_64 199 kB/s | 31 kB 00:00 2024-09-13T10:24:51.669 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:51.669 INFO:teuthology.orchestra.run.smithi173.stdout:Total 197 kB/s | 31 kB 00:00 2024-09-13T10:24:51.669 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:51.672 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:51.673 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:51.689 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:51.689 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:51.753 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:51.771 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:51.887 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:51.890 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:51.890 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:51.890 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:18.2.4-843.g2c01458a.el9 ceph 127 k 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:24:51.891 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:51.892 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 127 k 2024-09-13T10:24:51.892 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 456 k 2024-09-13T10:24:51.892 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:52.089 INFO:teuthology.orchestra.run.smithi060.stdout:librados-devel-18.2.4-843.g2c01458a.el9.x86_64. 642 kB/s | 127 kB 00:00 2024-09-13T10:24:52.090 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:52.090 INFO:teuthology.orchestra.run.smithi060.stdout:Total 637 kB/s | 127 kB 00:00 2024-09-13T10:24:52.090 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:52.097 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:52.097 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:52.158 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:52.158 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:52.162 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:02 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:52.225 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:52.293 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:52.294 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:52.294 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-13T10:24:52.294 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:52.294 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:18.2.4-843.g2c01458a.el9 ceph 31 k 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 31 k 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 139 k 2024-09-13T10:24:52.295 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:52.313 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:52.413 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:52.450 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-18.2.4-843.g2c01458a.el9.x86_64 199 kB/s | 31 kB 00:00 2024-09-13T10:24:52.451 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:52.451 INFO:teuthology.orchestra.run.smithi138.stdout:Total 197 kB/s | 31 kB 00:00 2024-09-13T10:24:52.451 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:52.455 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:52.455 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:52.464 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:52.464 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:52.465 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:52.465 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:52.472 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:52.473 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:52.562 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:52.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librados2 2024-09-13T10:24:52.679 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:53.099 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:53.099 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:53.099 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:53.099 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:53.100 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:53.100 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:53.204 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:53.256 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:53.257 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:53.294 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:53.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install libcephfs2 2024-09-13T10:24:53.335 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:53.337 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:53.337 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:53.418 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install librbd1 2024-09-13T10:24:53.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-09-13T10:24:53.907 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:03 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:53.974 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:53.989 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:03 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:54.001 INFO:teuthology.orchestra.run.smithi060.stdout:Package libcephfs2-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:54.046 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:54.047 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:54.047 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:54.063 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:54.079 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:54.105 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:54.106 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:54.106 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:54.121 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:54.122 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:54.122 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:54.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install libcephfs-devel 2024-09-13T10:24:54.180 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rados 2024-09-13T10:24:54.202 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-09-13T10:24:54.710 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:54.738 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:54.762 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:54.828 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:54.846 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:18.2.4-843.g2c01458a.el9 ceph 31 k 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:54.847 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:24:54.848 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:24:54.848 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:24:54.848 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:54.848 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 31 k 2024-09-13T10:24:54.848 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 139 k 2024-09-13T10:24:54.848 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:24:54.851 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:54.869 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:54.870 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:54.871 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:54.893 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:54.894 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:54.894 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:54.938 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rgw 2024-09-13T10:24:54.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-09-13T10:24:55.002 INFO:teuthology.orchestra.run.smithi060.stdout:libcephfs-devel-18.2.4-843.g2c01458a.el9.x86_64 202 kB/s | 31 kB 00:00 2024-09-13T10:24:55.003 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:55.003 INFO:teuthology.orchestra.run.smithi060.stdout:Total 201 kB/s | 31 kB 00:00 2024-09-13T10:24:55.003 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:24:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:24:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:24:55.026 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:24:55.026 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:24:55.113 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:24:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:55.492 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:55.522 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:55.582 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:55.606 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:55.612 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:55.624 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:55.625 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:55.625 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:55.654 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:55.655 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:55.655 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:55.697 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-cephfs 2024-09-13T10:24:55.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-09-13T10:24:55.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:55.841 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:55.841 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:24:55.842 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:55.842 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:24:55.842 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:56.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librados2 2024-09-13T10:24:56.260 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:56.300 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:56.349 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:56.386 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:56.391 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:56.392 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:56.392 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:56.428 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:56.429 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:56.429 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:56.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install python3-rbd 2024-09-13T10:24:56.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-09-13T10:24:56.625 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:56.717 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:56.763 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:56.764 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:56.764 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:56.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librbd1 2024-09-13T10:24:57.023 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:57.064 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:57.113 INFO:teuthology.orchestra.run.smithi173.stdout:Package python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:57.153 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:57.155 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:57.156 INFO:teuthology.orchestra.run.smithi173.stdout:Nothing to do. 2024-09-13T10:24:57.156 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:57.195 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:57.196 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:57.196 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:57.221 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-fuse 2024-09-13T10:24:57.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-09-13T10:24:57.415 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:57.509 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:57.552 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:57.553 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:57.554 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:57.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rados 2024-09-13T10:24:57.787 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:57.829 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:07 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:57.918 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:57.919 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:24:57.919 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:57.919 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-13T10:24:57.919 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:57.919 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:24:57.919 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 87 k 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 87 k 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 227 k 2024-09-13T10:24:57.920 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:24:57.960 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-13T10:24:57.961 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:58.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-09-13T10:24:58.112 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-fuse-18.2.4-843.g2c01458a.el9.x86_64.rpm 451 kB/s | 87 kB 00:00 2024-09-13T10:24:58.113 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:58.113 INFO:teuthology.orchestra.run.smithi173.stdout:Total 448 kB/s | 87 kB 00:00 2024-09-13T10:24:58.113 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:24:58.120 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:24:58.120 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:24:58.177 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:24:58.178 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:24:58.200 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:58.294 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rados-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:58.318 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:24:58.341 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:58.341 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:58.341 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:58.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rgw 2024-09-13T10:24:58.433 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:58.610 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:58.744 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:24:58.744 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 87 k 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:24:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:58.748 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 87 k 2024-09-13T10:24:58.749 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 227 k 2024-09-13T10:24:58.749 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:24:58.812 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:58.954 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-18.2.4-843.g2c01458a.el9.x86_64.rpm 416 kB/s | 87 kB 00:00 2024-09-13T10:24:58.955 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:24:58.955 INFO:teuthology.orchestra.run.smithi138.stdout:Total 413 kB/s | 87 kB 00:00 2024-09-13T10:24:58.955 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:24:58.962 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:24:58.962 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:24:59.002 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:59.020 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:24:59.020 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:24:59.088 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:59.088 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:59.088 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:24:59.089 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:59.089 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:24:59.089 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:24:59.097 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rgw-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:59.141 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:59.142 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:59.142 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:24:59.151 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:24:59.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-cephfs 2024-09-13T10:24:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:59.338 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-mirror 2024-09-13T10:24:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:59.798 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:09 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:24:59.890 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-cephfs-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:24:59.893 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:24:59.893 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:59.893 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:24:59.894 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:24:59.894 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:24:59.894 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:24:59.900 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:10 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:24:59.934 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:24:59.936 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:24:59.936 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:25:00.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rbd 2024-09-13T10:25:00.034 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repo Size 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.0 M 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:00.035 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:25:00.036 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:25:00.036 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:25:00.036 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:00.036 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 3.0 M 2024-09-13T10:25:00.036 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 12 M 2024-09-13T10:25:00.036 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:25:00.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-09-13T10:25:00.399 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-mirror-18.2.4-843.g2c01458a.el9.x86_64.rpm 8.2 MB/s | 3.0 MB 00:00 2024-09-13T10:25:00.400 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:00.401 INFO:teuthology.orchestra.run.smithi173.stdout:Total 8.2 MB/s | 3.0 MB 00:00 2024-09-13T10:25:00.401 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:25:00.409 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:25:00.409 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:25:00.488 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:25:00.489 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:25:00.582 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:10 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:25:00.675 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rbd-2:18.2.4-843.g2c01458a.el9.x86_64 is already installed. 2024-09-13T10:25:00.713 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:10 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:25:00.720 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:25:00.722 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-13T10:25:00.722 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:25:00.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-fuse 2024-09-13T10:25:00.849 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.0 M 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:25:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:00.851 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.0 M 2024-09-13T10:25:00.851 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 12 M 2024-09-13T10:25:00.851 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:25:00.919 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:25:01.085 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:01.108 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:01.109 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:25:01.109 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T10:25:01.109 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T10:25:01.109 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T10:25:01.109 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:01.202 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-18.2.4-843.g2c01458a.el9.x86_64.rpm 8.5 MB/s | 3.0 MB 00:00 2024-09-13T10:25:01.203 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:01.203 INFO:teuthology.orchestra.run.smithi138.stdout:Total 8.5 MB/s | 3.0 MB 00:00 2024-09-13T10:25:01.203 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:25:01.212 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:25:01.212 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:25:01.296 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:25:01.296 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:25:01.390 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:25:01.527 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:18.2.4-843.g2c01458a.el9 ceph 87 k 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:01.528 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:25:01.529 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:01.529 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:25:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:01.529 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 87 k 2024-09-13T10:25:01.529 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 227 k 2024-09-13T10:25:01.529 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:25:01.730 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-fuse-18.2.4-843.g2c01458a.el9.x86_64.rpm 431 kB/s | 87 kB 00:00 2024-09-13T10:25:01.731 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:01.731 INFO:teuthology.orchestra.run.smithi060.stdout:Total 428 kB/s | 87 kB 00:00 2024-09-13T10:25:01.731 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:25:01.738 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:25:01.739 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:25:01.747 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:25:01.799 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:25:01.800 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:25:01.820 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:01.850 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:01.850 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:25:01.850 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T10:25:01.850 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T10:25:01.850 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T10:25:01.850 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:25:02.037 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:02.145 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:02.145 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:02.146 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:25:02.146 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:02.146 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:02.146 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:25:02.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install rbd-nbd 2024-09-13T10:25:02.393 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:02.641 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:02.641 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:02.641 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:25:02.641 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:02.641 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:02.641 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:25:02.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-mirror 2024-09-13T10:25:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:02.928 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:25:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:02.928 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:25:02.964 INFO:teuthology.orchestra.run.smithi173.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Sep 2024 10:23:49 AM UTC. 2024-09-13T10:25:03.100 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-09-13T10:25:03.101 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:25:03.101 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2024-09-13T10:25:03.101 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:25:03.102 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-09-13T10:25:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 172 k 2024-09-13T10:25:03.102 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:03.102 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-09-13T10:25:03.102 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2024-09-13T10:25:03.103 INFO:teuthology.orchestra.run.smithi173.stdout:Install 1 Package 2024-09-13T10:25:03.103 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:03.103 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 172 k 2024-09-13T10:25:03.103 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 494 k 2024-09-13T10:25:03.103 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-09-13T10:25:03.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-09-13T10:25:03.313 INFO:teuthology.orchestra.run.smithi173.stdout:rbd-nbd-18.2.4-843.g2c01458a.el9.x86_64.rpm 816 kB/s | 172 kB 00:00 2024-09-13T10:25:03.313 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:03.314 INFO:teuthology.orchestra.run.smithi173.stdout:Total 809 kB/s | 172 kB 00:00 2024-09-13T10:25:03.314 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-09-13T10:25:03.321 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-09-13T10:25:03.321 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-09-13T10:25:03.379 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-09-13T10:25:03.379 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-09-13T10:25:03.435 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:25:03.532 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-09-13T10:25:03.575 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:25:03.575 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:03.575 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:18.2.4-843.g2c01458a.el9 ceph 3.0 M 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:03.576 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 3.0 M 2024-09-13T10:25:03.577 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 12 M 2024-09-13T10:25:03.577 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:25:03.625 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:03.729 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:25:03.861 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 172 k 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-13T10:25:03.862 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-13T10:25:03.863 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-13T10:25:03.863 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:03.863 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 172 k 2024-09-13T10:25:03.863 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 494 k 2024-09-13T10:25:03.863 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-13T10:25:03.918 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-mirror-18.2.4-843.g2c01458a.el9.x86_64.rpm 8.8 MB/s | 3.0 MB 00:00 2024-09-13T10:25:03.918 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:03.918 INFO:teuthology.orchestra.run.smithi060.stdout:Total 8.7 MB/s | 3.0 MB 00:00 2024-09-13T10:25:03.919 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:25:03.928 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:25:03.928 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:25:04.012 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:25:04.013 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:25:04.028 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:04.066 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-18.2.4-843.g2c01458a.el9.x86_64.rpm 846 kB/s | 172 kB 00:00 2024-09-13T10:25:04.066 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:04.067 INFO:teuthology.orchestra.run.smithi138.stdout:Total 840 kB/s | 172 kB 00:00 2024-09-13T10:25:04.067 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-13T10:25:04.074 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-13T10:25:04.075 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-13T10:25:04.132 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-13T10:25:04.133 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-13T10:25:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-13T10:25:04.341 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:04.341 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:04.341 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-09-13T10:25:04.341 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:04.341 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:25:04.342 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-09-13T10:25:04.387 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:25:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:04.544 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T10:25:04.545 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T10:25:04.545 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T10:25:04.545 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T10:25:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:04.596 DEBUG:teuthology.parallel:result is None 2024-09-13T10:25:04.793 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:05.063 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:05.063 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:05.063 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-13T10:25:05.063 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:05.063 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:25:05.063 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-13T10:25:05.330 DEBUG:teuthology.parallel:result is None 2024-09-13T10:25:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:05.598 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:25:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:05.598 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:25:05.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-nbd 2024-09-13T10:25:06.433 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:16 ago on Fri 13 Sep 2024 10:23:50 AM UTC. 2024-09-13T10:25:06.574 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-13T10:25:06.575 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:06.575 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-13T10:25:06.575 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:06.575 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-13T10:25:06.575 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:18.2.4-843.g2c01458a.el9 ceph 172 k 2024-09-13T10:25:06.575 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 172 k 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 494 k 2024-09-13T10:25:06.576 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-13T10:25:06.788 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-nbd-18.2.4-843.g2c01458a.el9.x86_64.rpm 812 kB/s | 172 kB 00:00 2024-09-13T10:25:06.789 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:25:06.790 INFO:teuthology.orchestra.run.smithi060.stdout:Total 803 kB/s | 172 kB 00:00 2024-09-13T10:25:06.790 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-13T10:25:06.799 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-13T10:25:06.799 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-13T10:25:06.860 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-13T10:25:06.860 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-13T10:25:07.027 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-13T10:25:07.137 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:07.525 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 1/1 2024-09-13T10:25:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:07.790 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-13T10:25:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:18.2.4-843.g2c01458a.el9.x86_64 2024-09-13T10:25:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:25:07.790 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-13T10:25:08.006 DEBUG:teuthology.parallel:result is None 2024-09-13T10:25:08.006 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:25:08.006 INFO:teuthology.packaging:ref: None 2024-09-13T10:25:08.007 INFO:teuthology.packaging:tag: None 2024-09-13T10:25:08.007 INFO:teuthology.packaging:branch: reef 2024-09-13T10:25:08.007 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:25:08.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T10:25:08.248 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T10:25:08.271 INFO:teuthology.orchestra.run.smithi060.stdout:18.2.4-843.g2c01458a.el9 2024-09-13T10:25:08.272 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-843.g2c01458a.el9 2024-09-13T10:25:08.272 INFO:teuthology.task.install:The correct ceph version 18.2.4-843.g2c01458a is installed. 2024-09-13T10:25:08.274 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:25:08.274 INFO:teuthology.packaging:ref: None 2024-09-13T10:25:08.274 INFO:teuthology.packaging:tag: None 2024-09-13T10:25:08.274 INFO:teuthology.packaging:branch: reef 2024-09-13T10:25:08.274 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:25:08.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T10:25:08.510 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T10:25:08.535 INFO:teuthology.orchestra.run.smithi138.stdout:18.2.4-843.g2c01458a.el9 2024-09-13T10:25:08.536 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-843.g2c01458a.el9 2024-09-13T10:25:08.536 INFO:teuthology.task.install:The correct ceph version 18.2.4-843.g2c01458a is installed. 2024-09-13T10:25:08.537 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T10:25:08.538 INFO:teuthology.packaging:ref: None 2024-09-13T10:25:08.538 INFO:teuthology.packaging:tag: None 2024-09-13T10:25:08.538 INFO:teuthology.packaging:branch: reef 2024-09-13T10:25:08.538 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:25:08.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T10:25:08.805 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T10:25:08.831 INFO:teuthology.orchestra.run.smithi173.stdout:18.2.4-843.g2c01458a.el9 2024-09-13T10:25:08.832 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-843.g2c01458a.el9 2024-09-13T10:25:08.832 INFO:teuthology.task.install:The correct ceph version 18.2.4-843.g2c01458a is installed. 2024-09-13T10:25:08.834 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T10:25:08.834 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:25:08.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:25:08.868 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:25:08.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:25:08.900 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:25:08.900 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:25:08.931 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T10:25:08.931 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:25:08.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:25:08.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:25:09.034 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:25:09.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:25:09.063 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:25:09.131 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:25:09.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:25:09.162 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:25:09.232 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T10:25:09.232 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:25:09.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:25:09.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:25:09.334 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:25:09.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:25:09.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:25:09.432 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:25:09.432 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:25:09.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:25:09.530 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T10:25:09.530 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:25:09.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:25:09.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:25:09.631 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:25:09.631 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:25:09.661 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:25:09.730 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:25:09.731 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:25:09.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:25:09.829 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-13T10:25:09.941 INFO:tasks.cephadm:Config: {'conf': {'mon': {'auth allow insecure global id reclaim': True, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'filestore odsync write': True, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', '\\(MON_DOWN\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'} 2024-09-13T10:25:09.941 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:25:09.942 INFO:tasks.cephadm:Cluster fsid is 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:25:09.942 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-13T10:25:09.942 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.60', 'mon.b': '172.21.15.138', 'mon.c': '172.21.15.173'} 2024-09-13T10:25:09.942 INFO:tasks.cephadm:First mon is mon.a on smithi060 2024-09-13T10:25:09.942 INFO:tasks.cephadm:First mgr is y 2024-09-13T10:25:09.942 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-13T10:25:09.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2024-09-13T10:25:09.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-09-13T10:25:10.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-09-13T10:25:10.034 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-13T10:25:10.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:25:10.174 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83028/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4697-gd58f7306', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4697.gd58f7306'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-12 23:59:16.034150', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-aclamk-cbt-combined', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2024-09-13T10:25:10.311 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-aclamk-cbt-combined, sha1 d58f73069a8157a9ace12a0247ee3e8497707ea9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:25:10.312 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/x86_64/flavors/default/cephadm 2024-09-13T10:25:10.312 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/x86_64/flavors/default/cephadm 2024-09-13T10:25:10.312 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:10.707 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-r--r--. 1 ubuntu ubuntu 802516 Sep 13 10:25 /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:10.707 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.089 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 802516 Sep 13 10:25 /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.090 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.473 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 802516 Sep 13 10:25 /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.474 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.494 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.514 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T10:25:11.546 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 on all hosts... 2024-09-13T10:25:11.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 pull 2024-09-13T10:25:11.549 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 pull 2024-09-13T10:25:11.557 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 pull 2024-09-13T10:25:13.222 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9... 2024-09-13T10:25:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9... 2024-09-13T10:25:13.320 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9... 2024-09-13T10:25:48.590 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:25:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev)", 2024-09-13T10:25:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd", 2024-09-13T10:25:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-09-13T10:25:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f" 2024-09-13T10:25:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:25:48.591 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:26:03.924 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:26:03.925 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev)", 2024-09-13T10:26:03.925 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd", 2024-09-13T10:26:03.925 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-09-13T10:26:03.925 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f" 2024-09-13T10:26:03.925 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:26:03.925 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:26:04.214 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:26:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev)", 2024-09-13T10:26:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd", 2024-09-13T10:26:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2024-09-13T10:26:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f" 2024-09-13T10:26:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:26:04.215 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:26:04.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2024-09-13T10:26:04.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-09-13T10:26:04.317 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-09-13T10:26:04.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2024-09-13T10:26:04.377 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-09-13T10:26:04.405 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-09-13T10:26:04.434 INFO:tasks.cephadm:Writing seed config... 2024-09-13T10:26:04.435 INFO:tasks.cephadm: override: [mon] auth allow insecure global id reclaim = True 2024-09-13T10:26:04.435 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-13T10:26:04.435 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] mon scrub interval = 300 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [global] mon client directed command retry = 5 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [global] ms inject delay max = 1 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [global] ms inject delay probability = 0.005 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [global] ms inject delay type = osd 2024-09-13T10:26:04.436 INFO:tasks.cephadm: override: [global] ms inject internal delays = 0.002 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [global] ms inject socket failures = 2500 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [global] osd_pool_default_min_size = 2 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [global] osd_pool_default_size = 2 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] bluestore zero block detection = True 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] filestore odsync write = True 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] osd backoff on degraded = True 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] osd backoff on peering = True 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] osd blocked scrub grace period = 3600 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] osd debug verify cached snaps = True 2024-09-13T10:26:04.437 INFO:tasks.cephadm: override: [osd] osd debug verify missing on start = True 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd max backfills = 2 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd max markdown count = 1000 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd mclock override recovery settings = True 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd mclock profile = high_recovery_ops 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd op queue = debug_random 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd op queue cut off = debug_random 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-13T10:26:04.438 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 0.5 2024-09-13T10:26:04.439 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:26:04.439 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-13T10:26:04.455 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 74376cb6-71ba-11ef-bceb-c7b262605968 mon client directed command retry = 5 mon election default strategy = 1 ms inject delay max = 1 ms inject delay probability = 0.005 ms inject delay type = osd ms inject internal delays = 0.002 ms inject socket failures = 2500 osd_pool_default_min_size = 2 osd_pool_default_size = 2 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = True osd debug verify missing on start = True osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore zero block detection = True debug ms = 1 debug osd = 20 filestore odsync write = True osd backoff on degraded = True osd backoff on peering = True osd blocked scrub grace period = 3600 osd max backfills = 2 osd max markdown count = 1000 osd mclock override recovery settings = True osd scrub min interval = 60 osd snap trim sleep = 0.5 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false auth allow insecure global id reclaim = True debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 mon scrub interval = 300 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-13T10:26:04.455 DEBUG:teuthology.orchestra.run.smithi060:mon.a> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a.service 2024-09-13T10:26:04.498 DEBUG:teuthology.orchestra.run.smithi060:mgr.y> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y.service 2024-09-13T10:26:04.541 INFO:tasks.cephadm:Bootstrapping... 2024-09-13T10:26:04.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 -v bootstrap --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.60 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:26:04.750 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-13T10:26:04.751 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9', '-v', 'bootstrap', '--fsid', '74376cb6-71ba-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.60', '--skip-admin-label'] 2024-09-13T10:26:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-13T10:26:04.751 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying podman|docker is present... 2024-09-13T10:26:04.775 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 5.2.2 2024-09-13T10:26:04.776 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying lvm2 is present... 2024-09-13T10:26:04.776 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying time synchronization is in place... 2024-09-13T10:26:04.783 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T10:26:04.783 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T10:26:04.790 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T10:26:04.790 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-13T10:26:04.797 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-09-13T10:26:04.803 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-09-13T10:26:04.804 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-09-13T10:26:04.804 INFO:teuthology.orchestra.run.smithi060.stdout:Repeating the final host check... 2024-09-13T10:26:04.826 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 5.2.2 2024-09-13T10:26:04.827 INFO:teuthology.orchestra.run.smithi060.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-13T10:26:04.827 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl is present 2024-09-13T10:26:04.827 INFO:teuthology.orchestra.run.smithi060.stdout:lvcreate is present 2024-09-13T10:26:04.833 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T10:26:04.833 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T10:26:04.840 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T10:26:04.840 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-13T10:26:04.847 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-09-13T10:26:04.853 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-09-13T10:26:04.853 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Host looks OK 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Cluster fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Acquiring lock 140024728173632 on /run/cephadm/74376cb6-71ba-11ef-bceb-c7b262605968.lock 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Lock 140024728173632 acquired on /run/cephadm/74376cb6-71ba-11ef-bceb-c7b262605968.lock 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 3300 ... 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 6789 ... 2024-09-13T10:26:04.854 INFO:teuthology.orchestra.run.smithi060.stdout:Base mon IP(s) is [172.21.15.60:3300, 172.21.15.60:6789], mon addrv is [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-09-13T10:26:04.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.60 metric 100 2024-09-13T10:26:04.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.60 metric 100 2024-09-13T10:26:04.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-13T10:26:04.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-13T10:26:04.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1787sec hoplimit 64 pref medium 2024-09-13T10:26:04.863 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-13T10:26:04.863 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-13T10:26:04.863 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T10:26:04.863 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-13T10:26:04.863 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-13T10:26:04.863 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T10:26:04.864 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-09-13T10:26:04.864 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-09-13T10:26:04.864 INFO:teuthology.orchestra.run.smithi060.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-13T10:26:04.864 INFO:teuthology.orchestra.run.smithi060.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-13T10:26:04.865 INFO:teuthology.orchestra.run.smithi060.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9... 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9... 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Getting image source signatures 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:4f1e227cf18e81f5e2a5c9a48a6b57055253289fec8a5452649f17f76ee9baf7 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying config sha256:fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd 2024-09-13T10:26:05.326 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-13T10:26:06.272 INFO:teuthology.orchestra.run.smithi060.stdout:ceph: stdout ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev) 2024-09-13T10:26:06.272 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph version: ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev) 2024-09-13T10:26:06.273 INFO:teuthology.orchestra.run.smithi060.stdout:Extracting ceph user uid/gid from container image... 2024-09-13T10:26:07.503 INFO:teuthology.orchestra.run.smithi060.stdout:stat: stdout 167 167 2024-09-13T10:26:07.503 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial keys... 2024-09-13T10:26:08.459 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQA/E+RmZCSjMhAAA9kirhqASiEUWbQS9WBGxg== 2024-09-13T10:26:16.829 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQBCE+Rmk2MQHBAA+YohtnZ2v7hsgmz5JOtG8g== 2024-09-13T10:26:20.712 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQBLE+RmMyySKBAAdhefH3OEQq5CV+Ok0NMvOw== 2024-09-13T10:26:20.713 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial monmap... 2024-09-13T10:26:21.676 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T10:26:21.676 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-13T10:26:21.676 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:21.676 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T10:26:21.676 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool for a [v2:172.21.15.60:3300,v1:172.21.15.60:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T10:26:21.676 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-09-13T10:26:21.677 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: set fsid to 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:21.677 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T10:26:21.677 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:26:21.677 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mon... 2024-09-13T10:26:22.748 INFO:teuthology.orchestra.run.smithi060.stdout:create mon.a on 2024-09-13T10:26:22.953 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-13T10:26:23.098 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-13T10:26:23.286 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-74376cb6-71ba-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-74376cb6-71ba-11ef-bceb-c7b262605968.target. 2024-09-13T10:26:23.287 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-74376cb6-71ba-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-74376cb6-71ba-11ef-bceb-c7b262605968.target. 2024-09-13T10:26:23.530 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a 2024-09-13T10:26:23.530 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-13T10:26:23.702 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-74376cb6-71ba-11ef-bceb-c7b262605968.target.wants/ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-74376cb6-71ba-11ef-bceb-c7b262605968@.service. 2024-09-13T10:26:24.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:23 smithi060 podman[26983]: 2024-09-13 10:26:23.839776605 +0000 UTC m=+0.016517382 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:26:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T10:26:24.324 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-13T10:26:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T10:26:24.332 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-13T10:26:24.332 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-13T10:26:24.332 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T10:26:24.332 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon to start... 2024-09-13T10:26:24.332 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon... 2024-09-13T10:26:24.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:24 smithi060 podman[26983]: 2024-09-13 10:26:24.241090607 +0000 UTC m=+0.417831383 container init 9a15bdecf942b90363369bd87a032c2e8cdb1a2765b46cf9836aa9aab4a26330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:26:24.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:24 smithi060 podman[26983]: 2024-09-13 10:26:24.244582987 +0000 UTC m=+0.421323760 container start 9a15bdecf942b90363369bd87a032c2e8cdb1a2765b46cf9836aa9aab4a26330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-13T10:26:24.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:24 smithi060 ceph-mon[27004]: mkfs 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:24.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:24 smithi060 ceph-mon[27004]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T10:26:24.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:24 smithi060 bash[26983]: 9a15bdecf942b90363369bd87a032c2e8cdb1a2765b46cf9836aa9aab4a26330 2024-09-13T10:26:24.452 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:24 smithi060 systemd[1]: Started Ceph mon.a for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:26:25.461 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout cluster: 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout id: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout services: 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.569493s) 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout data: 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-13T10:26:25.462 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pgs: 2024-09-13T10:26:25.463 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:25.463 INFO:teuthology.orchestra.run.smithi060.stdout:mon is available 2024-09-13T10:26:25.463 INFO:teuthology.orchestra.run.smithi060.stdout:Assimilating anything we can from ceph.conf... 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: monmap epoch 1 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: last_changed 2024-09-13T10:26:21.083088+0000 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: min_mon_release 19 (squid) 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: election_strategy: 1 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: fsmap 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: mgrmap e1: no daemons active 2024-09-13T10:26:25.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:25 smithi060 ceph-mon[27004]: from='client.? 172.21.15.60:0/2078873287' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-13T10:26:26.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:26 smithi060 ceph-mon[27004]: from='client.? 172.21.15.60:0/258500493' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-13T10:26:26.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:26 smithi060 ceph-mon[27004]: from='client.? 172.21.15.60:0/258500493' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-13T10:26:27.022 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:27.022 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:27.023 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T10:26:27.024 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T10:26:27.024 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T10:26:27.024 INFO:teuthology.orchestra.run.smithi060.stdout:Generating new minimal ceph.conf... 2024-09-13T10:26:27.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:27 smithi060 ceph-mon[27004]: from='client.? 172.21.15.60:0/620592224' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:26:28.270 INFO:teuthology.orchestra.run.smithi060.stdout:Restarting the monitor... 2024-09-13T10:26:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:28 smithi060 systemd[1]: Stopping Ceph mon.a for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:26:28.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:28 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a[27000]: 2024-09-13T10:26:28.551+0000 7fd4894d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:26:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:28 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a[27000]: 2024-09-13T10:26:28.551+0000 7fd4894d2640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-13T10:26:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:28 smithi060 podman[27249]: 2024-09-13 10:26:28.675338136 +0000 UTC m=+0.220417984 container died 9a15bdecf942b90363369bd87a032c2e8cdb1a2765b46cf9836aa9aab4a26330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux ) 2024-09-13T10:26:29.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 podman[27249]: 2024-09-13 10:26:29.042808978 +0000 UTC m=+0.587888824 container cleanup 9a15bdecf942b90363369bd87a032c2e8cdb1a2765b46cf9836aa9aab4a26330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=) 2024-09-13T10:26:29.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 bash[27249]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a 2024-09-13T10:26:29.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 podman[27261]: 2024-09-13 10:26:29.25227613 +0000 UTC m=+0.567792161 container remove 9a15bdecf942b90363369bd87a032c2e8cdb1a2765b46cf9836aa9aab4a26330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-13T10:26:29.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 systemd[1]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T10:26:29.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 systemd[1]: Stopped Ceph mon.a for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:26:29.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 systemd[1]: Starting Ceph mon.a for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:26:29.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 podman[27356]: 2024-09-13 10:26:29.588328683 +0000 UTC m=+0.016273473 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:26:29.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 podman[27356]: 2024-09-13 10:26:29.728541953 +0000 UTC m=+0.156486738 container create 3f6e391b81b9286ea28c7d6811162eba8e0defd39479326945d82e56e74718e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-13T10:26:29.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 podman[27356]: 2024-09-13 10:26:29.922422525 +0000 UTC m=+0.350367305 container init 3f6e391b81b9286ea28c7d6811162eba8e0defd39479326945d82e56e74718e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-13T10:26:29.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 podman[27356]: 2024-09-13 10:26:29.925578302 +0000 UTC m=+0.353523091 container start 3f6e391b81b9286ea28c7d6811162eba8e0defd39479326945d82e56e74718e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-13T10:26:30.019 INFO:teuthology.orchestra.run.smithi060.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-13T10:26:30.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T10:26:30.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev), process ceph-mon, pid 2 2024-09-13T10:26:30.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: pidfile_write: ignore empty --pid-file 2024-09-13T10:26:30.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: load: jerasure load: lrc 2024-09-13T10:26:30.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: RocksDB version: 7.9.2 2024-09-13T10:26:30.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Git sha 0 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Compile date 2024-09-12 20:46:14 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: DB SUMMARY 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: DB Session ID: EL2ODGR6UJ718LADJBV6 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: CURRENT file: CURRENT 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101059 ; 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.error_if_exists: 0 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.create_if_missing: 0 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.paranoid_checks: 1 2024-09-13T10:26:30.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.env: 0x560a44c91160 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.info_log: 0x560a46b25880 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.statistics: (nil) 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.use_fsync: 0 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_log_file_size: 0 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T10:26:30.255 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.allow_fallocate: 1 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.use_direct_reads: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.db_log_dir: 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.wal_dir: 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T10:26:30.256 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.write_buffer_manager: 0x560a46b28e60 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T10:26:30.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.unordered_write: 0 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.row_cache: None 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.wal_filter: None 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.two_write_queues: 0 2024-09-13T10:26:30.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.wal_compression: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.atomic_flush: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.log_readahead_size: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T10:26:30.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T10:26:30.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_background_jobs: 2 2024-09-13T10:26:30.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_background_compactions: -1 2024-09-13T10:26:30.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_subcompactions: 1 2024-09-13T10:26:30.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T10:26:30.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T10:26:30.261 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_open_files: -1 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_background_flushes: -1 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Compression algorithms supported: 2024-09-13T10:26:30.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kZSTD supported: 0 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kXpressCompression supported: 0 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kBZip2Compression supported: 0 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kLZ4Compression supported: 1 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kZlibCompression supported: 1 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: kSnappyCompression supported: 1 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.merge_operator: 2024-09-13T10:26:30.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_filter: None 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560a46b254e0) 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_type: 0 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_index_type: 0 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_shortening: 1 2024-09-13T10:26:30.264 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: checksum: 4 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: no_block_cache: 0 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache: 0x560a46b18850 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_options: 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: capacity : 536870912 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_shard_bits : 4 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: strict_capacity_limit : 0 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_compressed: (nil) 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: persistent_cache: (nil) 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size: 4096 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size_deviation: 10 2024-09-13T10:26:30.265 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_restart_interval: 16 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_block_restart_interval: 1 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: metadata_block_size: 4096 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: partition_filters: 0 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: use_delta_encoding: 1 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: filter_policy: bloomfilter 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: whole_key_filtering: 1 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: verify_compression: 0 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: format_version: 5 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_index_compression: 1 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_align: 0 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: max_auto_readahead_size: 262144 2024-09-13T10:26:30.266 INFO:journalctl@ceph.mon.a.smithi060.stdout: prepopulate_block_cache: 0 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout: initial_auto_readahead_size: 8192 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression: NoCompression 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.num_levels: 7 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T10:26:30.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T10:26:30.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T10:26:30.269 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T10:26:30.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.inplace_update_support: 0 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T10:26:30.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.bloom_locality: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.max_successive_merges: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.ttl: 2592000 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.enable_blob_files: false 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.min_blob_size: 0 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T10:26:30.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9840f352-ac48-4139-b0d9-fed56464caf2 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726223189952614, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-13T10:26:30.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726223189953657, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96479, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 307, "table_properties": {"data_size": 94468, "index_size": 267, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13255, "raw_average_key_size": 49, "raw_value_size": 87214, "raw_average_value_size": 327, "num_data_blocks": 12, "num_entries": 266, "num_filter_entries": 266, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726223189, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9840f352-ac48-4139-b0d9-fed56464caf2", "db_session_id": "EL2ODGR6UJ718LADJBV6", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726223189953756, "job": 1, "event": "recovery_finished"} 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560a46b48e00 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: DB pointer 0x560a46c48000 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** DB Stats ** 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T10:26:30.274 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: L0 2/0 96.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: Sum 2/0 96.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T10:26:30.275 INFO:journalctl@ceph.mon.a.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative compaction: 0.00 GB write, 27.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval compaction: 0.00 GB write, 27.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T10:26:30.276 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache BinnedLRUCache@0x560a46b18850#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: starting mon.a rank 0 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???) e1 preinit fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).mds e1 new map 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).mds e1 print_map 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: e1 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: btime 2024-09-13T10:26:24:281765+0000 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: legacy client fscid: -1 2024-09-13T10:26:30.277 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout: No filesystems configured 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:29 smithi060 ceph-mon[27375]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: monmap epoch 1 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: last_changed 2024-09-13T10:26:21.083088+0000 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: min_mon_release 19 (squid) 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: election_strategy: 1 2024-09-13T10:26:30.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:26:30.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: fsmap 2024-09-13T10:26:30.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T10:26:30.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 ceph-mon[27375]: mgrmap e1: no daemons active 2024-09-13T10:26:30.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 bash[27356]: 3f6e391b81b9286ea28c7d6811162eba8e0defd39479326945d82e56e74718e9 2024-09-13T10:26:30.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:30 smithi060 systemd[1]: Started Ceph mon.a for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:26:30.959 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-13T10:26:30.980 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:26:30.981 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mgr... 2024-09-13T10:26:30.981 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-13T10:26:30.981 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-13T10:26:31.272 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y 2024-09-13T10:26:31.273 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-13T10:26:31.445 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-74376cb6-71ba-11ef-bceb-c7b262605968.target.wants/ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-74376cb6-71ba-11ef-bceb-c7b262605968@.service. 2024-09-13T10:26:31.530 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 systemd[1]: Starting Ceph mgr.y for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:26:31.799 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:31 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1095100825' entity='client.admin' 2024-09-13T10:26:31.799 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 podman[27586]: 2024-09-13 10:26:31.67363569 +0000 UTC m=+0.092560997 container create 6f5f208bdaabd43b2892db1362472301dc8fdf394f01265131ac0767ead501b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-13T10:26:31.799 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 podman[27586]: 2024-09-13 10:26:31.597351919 +0000 UTC m=+0.016277235 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:26:31.992 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T10:26:31.992 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-13T10:26:31.999 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T10:26:31.999 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-13T10:26:31.999 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-13T10:26:31.999 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T10:26:32.008 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T10:26:32.008 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-13T10:26:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T10:26:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-13T10:26:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-13T10:26:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-13T10:26:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr to start... 2024-09-13T10:26:32.016 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr... 2024-09-13T10:26:32.144 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 podman[27586]: 2024-09-13 10:26:31.8927644 +0000 UTC m=+0.311689714 container init 6f5f208bdaabd43b2892db1362472301dc8fdf394f01265131ac0767ead501b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T10:26:32.144 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 podman[27586]: 2024-09-13 10:26:31.895564781 +0000 UTC m=+0.314490087 container start 6f5f208bdaabd43b2892db1362472301dc8fdf394f01265131ac0767ead501b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-13T10:26:32.145 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 bash[27586]: 6f5f208bdaabd43b2892db1362472301dc8fdf394f01265131ac0767ead501b4 2024-09-13T10:26:32.145 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:31 smithi060 systemd[1]: Started Ceph mgr.y for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:26:32.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:32 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/428121841' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-13T10:26:33.103 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:33.111 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:26:33.111 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "74376cb6-71ba-11ef-bceb-c7b262605968", 2024-09-13T10:26:33.111 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:33.112 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T10:26:33.113 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T10:26:33.114 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-09-13T10:26:24:281765+0000", 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:33.115 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T10:26:24.282409+0000", 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:26:33.116 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (1/15)... 2024-09-13T10:26:35.849 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:35 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1367772195' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "74376cb6-71ba-11ef-bceb-c7b262605968", 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T10:26:36.234 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-09-13T10:26:24:281765+0000", 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T10:26:36.235 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T10:26:24.282409+0000", 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:26:36.236 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (2/15)... 2024-09-13T10:26:37.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: Activating manager daemon y 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: mgrmap e2: y(active, starting, since 0.00332999s) 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: Manager daemon y is now available 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' 2024-09-13T10:26:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' 2024-09-13T10:26:37.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:37 smithi060 ceph-mon[27375]: from='mgr.14100 172.21.15.60:0/1198808497' entity='mgr.y' 2024-09-13T10:26:39.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:39 smithi060 ceph-mon[27375]: mgrmap e3: y(active, since 1.00769s) 2024-09-13T10:26:39.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:39 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2137521378' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-13T10:26:39.869 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "74376cb6-71ba-11ef-bceb-c7b262605968", 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T10:26:39.870 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T10:26:39.871 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T10:26:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-09-13T10:26:24:281765+0000", 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T10:26:39.876 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T10:26:24.282409+0000", 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:26:39.877 INFO:teuthology.orchestra.run.smithi060.stdout:mgr is available 2024-09-13T10:26:40.300 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:40 smithi060 ceph-mon[27375]: mgrmap e4: y(active, since 2s) 2024-09-13T10:26:41.116 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-13T10:26:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T10:26:41.118 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T10:26:41.118 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T10:26:41.118 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling cephadm module... 2024-09-13T10:26:41.456 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:41 smithi060 ceph-mon[27375]: from='client.14114 172.21.15.60:0/2729646260' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-13T10:26:42.493 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3067897599' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-13T10:26:42.493 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:42 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: ignoring --setuser ceph since I am not root 2024-09-13T10:26:42.494 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:42 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: ignoring --setgroup ceph since I am not root 2024-09-13T10:26:43.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:43 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3067897599' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-13T10:26:43.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:43 smithi060 ceph-mon[27375]: mgrmap e5: y(active, since 5s) 2024-09-13T10:26:44.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:26:44.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-13T10:26:44.053 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T10:26:44.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-13T10:26:44.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T10:26:44.054 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:26:44.054 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-09-13T10:26:44.054 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 5... 2024-09-13T10:26:44.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:44 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2089149468' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-13T10:26:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: Active manager daemon y restarted 2024-09-13T10:26:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: Activating manager daemon y 2024-09-13T10:26:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: osdmap e2: 0 total, 0 up, 0 in 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: mgrmap e6: y(active, starting, since 0.00308531s) 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T10:26:47.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:47 smithi060 ceph-mon[27375]: Manager daemon y is now available 2024-09-13T10:26:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:26:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-13T10:26:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T10:26:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:26:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 5 is available 2024-09-13T10:26:48.976 INFO:teuthology.orchestra.run.smithi060.stdout:Setting orchestrator backend to cephadm... 2024-09-13T10:26:49.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: mgrmap e7: y(active, since 1.00498s) 2024-09-13T10:26:49.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T10:26:49.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T10:26:49.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:49.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:49.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:49.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:49.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:26:49.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:26:49.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T10:26:49.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:49 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T10:26:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:50 smithi060 ceph-mon[27375]: Found migration_current of "None". Setting to last migration. 2024-09-13T10:26:50.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:50 smithi060 ceph-mon[27375]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:26:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:50 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:50 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:26:50.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:50 smithi060 ceph-mon[27375]: mgrmap e8: y(active, since 2s) 2024-09-13T10:26:51.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-13T10:26:51.329 INFO:teuthology.orchestra.run.smithi060.stdout:Generating ssh key... 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: 2024-09-13T10:26:51.341+0000 7f2a1fe4c640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:10:26:51] ENGINE Error in HTTPServer.serve 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: Traceback (most recent call last): 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self._connections.run(self.expiration_interval) 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self._run(expiration_interval) 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T10:26:51.601 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: s = self.context.wrap_socket( 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: return self.sslsocket_class._create( 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self.do_handshake() 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self._sslobj.do_handshake() 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T10:26:51.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: 2024-09-13T10:26:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-mon[27375]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:26:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-mon[27375]: [13/Sep/2024:10:26:51] ENGINE Bus STARTING 2024-09-13T10:26:51.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-mon[27375]: [13/Sep/2024:10:26:51] ENGINE Serving on http://172.21.15.60:8765 2024-09-13T10:26:51.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:26:52.325 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: Generating public/private rsa key pair. 2024-09-13T10:26:52.325 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: Your identification has been saved in /tmp/tmpd6at5vit/key 2024-09-13T10:26:52.325 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: Your public key has been saved in /tmp/tmpd6at5vit/key.pub 2024-09-13T10:26:52.325 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: The key fingerprint is: 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: SHA256:r0cyQyfiprcj5aphrOBVYCJYVrjv8uCsVWKRCJUQKTI ceph-74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: The key's randomart image is: 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: +---[RSA 3072]----+ 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |+=++. | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |E+o. | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |*oo+ | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |. +.. . o . | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: | o..o oSo | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: | o oo + +.. | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |. *o = =. | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |o*oo+ + .. | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: |oo++o+.o.. | 2024-09-13T10:26:52.326 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:26:51 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: +----[SHA256]-----+ 2024-09-13T10:26:52.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: [13/Sep/2024:10:26:51] ENGINE Serving on https://172.21.15.60:7150 2024-09-13T10:26:52.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: [13/Sep/2024:10:26:51] ENGINE Bus STARTED 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: [13/Sep/2024:10:26:51] ENGINE Error in HTTPServer.serve 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T10:26:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: mgrmap e9: y(active, since 4s) 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: Generating ssh key... 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:52.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:52 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:53.826 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:26:53.826 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-13T10:26:53.826 INFO:teuthology.orchestra.run.smithi060.stdout:Adding key to root@localhost authorized_keys... 2024-09-13T10:26:53.836 INFO:teuthology.orchestra.run.smithi060.stdout:Adding host smithi060... 2024-09-13T10:26:54.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:54 smithi060 ceph-mon[27375]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:26:55.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:55 smithi060 ceph-mon[27375]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:26:56.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:56 smithi060 ceph-mon[27375]: Deploying cephadm binary to smithi060 2024-09-13T10:26:59.000 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Added host 'smithi060' with addr '172.21.15.60' 2024-09-13T10:26:59.000 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mon service... 2024-09-13T10:26:59.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:59 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:26:59.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:59 smithi060 ceph-mon[27375]: Added host smithi060 2024-09-13T10:26:59.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:26:59 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:00.434 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-13T10:27:00.434 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mgr service... 2024-09-13T10:27:01.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:00 smithi060 ceph-mon[27375]: Saving service mon spec with placement count:5 2024-09-13T10:27:01.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:00 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:27:01.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:00 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:27:02.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:02 smithi060 ceph-mon[27375]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:02.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:02 smithi060 ceph-mon[27375]: Saving service mgr spec with placement count:2 2024-09-13T10:27:02.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:02 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:27:02.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:02 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:27:02.505 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-13T10:27:03.720 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:03 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:27:03.720 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:03 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/168132612' entity='client.admin' 2024-09-13T10:27:05.371 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling the dashboard module... 2024-09-13T10:27:05.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:05 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1641231819' entity='client.admin' 2024-09-13T10:27:05.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:05 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:27:06.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:06 smithi060 ceph-mon[27375]: from='mgr.14118 172.21.15.60:0/2034035369' entity='mgr.y' 2024-09-13T10:27:06.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:06 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/588600112' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-13T10:27:06.898 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:06 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: ignoring --setuser ceph since I am not root 2024-09-13T10:27:06.898 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:06 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: ignoring --setgroup ceph since I am not root 2024-09-13T10:27:07.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:07 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/588600112' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-13T10:27:07.812 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:07 smithi060 ceph-mon[27375]: mgrmap e10: y(active, since 19s) 2024-09-13T10:27:08.815 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:08 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2901242969' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-09-13T10:27:08.819 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 10... 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: Active manager daemon y restarted 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: Activating manager daemon y 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: osdmap e3: 0 total, 0 up, 0 in 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: mgrmap e11: y(active, starting, since 0.00319594s) 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-13T10:27:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-13T10:27:12.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-13T10:27:12.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:11 smithi060 ceph-mon[27375]: Manager daemon y is now available 2024-09-13T10:27:13.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:12 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:13.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:12 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-13T10:27:13.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:12 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-13T10:27:13.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:12 smithi060 ceph-mon[27375]: mgrmap e12: y(active, since 1.00636s) 2024-09-13T10:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-13T10:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-13T10:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T10:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-13T10:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 10 is available 2024-09-13T10:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-13T10:27:14.219 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:13 smithi060 ceph-mon[27375]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T10:27:14.219 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:13 smithi060 ceph-mon[27375]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T10:27:14.219 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: 2024-09-13T10:27:14.127+0000 7f246428e640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:10:27:14] ENGINE Error in HTTPServer.serve 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: Traceback (most recent call last): 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self._connections.run(self.expiration_interval) 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self._run(expiration_interval) 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: s = self.context.wrap_socket( 2024-09-13T10:27:14.220 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: return self.sslsocket_class._create( 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self.do_handshake() 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: self._sslobj.do_handshake() 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T10:27:14.221 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: 2024-09-13T10:27:15.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: [13/Sep/2024:10:27:13] ENGINE Bus STARTING 2024-09-13T10:27:15.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: mgrmap e13: y(active, since 2s) 2024-09-13T10:27:15.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: [13/Sep/2024:10:27:14] ENGINE Serving on http://172.21.15.60:8765 2024-09-13T10:27:15.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: [13/Sep/2024:10:27:14] ENGINE Serving on https://172.21.15.60:7150 2024-09-13T10:27:15.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: [13/Sep/2024:10:27:14] ENGINE Bus STARTED 2024-09-13T10:27:15.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: [13/Sep/2024:10:27:14] ENGINE Error in HTTPServer.serve 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-09-13T10:27:15.233 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:15.234 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:15.526 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUbD2sihDUetfZZ2xjUHZkrmNX9TwwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMzEwMjcxNFoXDTM0\nMDkxNDEwMjcxNFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjYwMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAmUerenE7NoV03+fgwmJYfFVyu5OUElWlaKpk\nMxmWdMz53/q39E5gU+HE8JCDbrgE2YHFCXxNCNfmDVb2Py/5+Vmd+cg1dvMp4q2Z\nWUBj+fKdgrGeF8pig7WOjWLrHKbtxhIXfKidbg5nmiLKs9DFf/rHRpjdvbGSqz8i\nIRFlBnFuQlulVOtUkJTD6EU0XgdTDebswnv2q+Lb4UuhFhzzTJ77Fx9gztehIP3e\nDg9Qz17W0d0/A+7hBJrl2+OMkG7QrRHnrvIuaeBn3CHstkJDtymk1z6o7/8Pr42q\n/FMQg++P61qKtoNe7Ei+w1YDg+H2HmaWL+heky0U3PNviT9MDdYjVaJ5CGHe7V2p\nbnmvXen5kQ3T8jwYj2wDubqVfkv6cuoAQ4nCwlEv2F1Z7jMizZ8tSnC0Ol612Luz\nTo572WBgRvJCmalAkfEM+qIsEFBsetMSI8lb/oPD6VzcIBQil3YrV6vROWzQicFc\nZ7CUoSBRX/aTHYfvxURcl2eUJ6taUaa90C61nsrwXd89F9U89VK2jry6+nTvMDvB\n9Z3h0kLbG3OylP+SRgjvcQKUK5D2++mTwQftIIrRqj4Lne3/5kRvLLYI+YjmuXuH\nXZSA/0l5DBcTKBFGyrcdUprdWx42GbeBuxpH5cHOzLlUeqmFkqBHgnpAwtdV4+F3\nLko2s3MCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDYwghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPPDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBB\nNcdZIrw9/+fkfAUnAesb1sIlx44DVWmbtzitq6B/Vo43Wmx0nXzS4oi4o1o4qOQk\nLArItL8wzihpVUUh8exNpEKX6jGXonK5hAcuYjx8mTuaVic4VNm0tt7uDV+61zkD\n3+JqwGLB/vT2AFgo4nSIurmFlGeDBTp8czdQzufR3u0Ka4yQw0/SiLe+/gqUZL+F\nfSW1fohtGAfxU3PeJERXKTMfQhZB7HGOlH7HRt49rpQQLpGOrQ5oeoB21PHt3dyM\nKKOwSjlQI4V7GhxzgELjG3HJl2h9Ow0DqOvS56ceuFSG1b8imXkrZSwAc980Y+pY\nT1h4FDFhq6Nc8OPKEmoBnourZn8OAeutMNBItGXeqV0bMyhxfK0BoSsibZ8GUGNx\nJK2Y30ert4gdmd9ppe5HO8VmxGXA01yVKShsaUXYGFYpnTcD7ptP2GDuY9bSHHma\nc0zG8QKM8b6g4Xh3gSMv1l40cBcaCnpuxd3bwzPukVdvgSFo0jNW6zsFrVeDAW1z\n83fX8zZ2M11fj6N4Vy83+f4RQmdLhjMN+P4yTlsl19TWua1leKZH9zaEslt9Gzya\nxoot+AooNmJGd57LhqIfvKWL2ZajTZnjWs4EhAwrhlVMtBV/GOsrFlTHtBmyfM0z\nxbJSDARARDkwcdHci6vuoVqZN6nNIQyXrnIAiPNraw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAmUerenE7NoV03+fgwmJYfFVyu5OUElWlaKpkMxmWdMz53/q3\n9E5gU+HE8JCDbrgE2YHFCXxNCNfmDVb2Py/5+Vmd+cg1dvMp4q2ZWUBj+fKdgrGe\nF8pig7WOjWLrHKbtxhIXfKidbg5nmiLKs9DFf/rHRpjdvbGSqz8iIRFlBnFuQlul\nVOtUkJTD6EU0XgdTDebswnv2q+Lb4UuhFhzzTJ77Fx9gztehIP3eDg9Qz17W0d0/\nA+7hBJrl2+OMkG7QrRHnrvIuaeBn3CHstkJDtymk1z6o7/8Pr42q/FMQg++P61qK\ntoNe7Ei+w1YDg+H2HmaWL+heky0U3PNviT9MDdYjVaJ5CGHe7V2pbnmvXen5kQ3T\n8jwYj2wDubqVfkv6cuoAQ4nCwlEv2F1Z7jMizZ8tSnC0Ol612LuzTo572WBgRvJC\nmalAkfEM+qIsEFBsetMSI8lb/oPD6VzcIBQil3YrV6vROWzQicFcZ7CUoSBRX/aT\nHYfvxURcl2eUJ6taUaa90C61nsrwXd89F9U89VK2jry6+nTvMDvB9Z3h0kLbG3Oy\nlP+SRgjvcQKUK5D2++mTwQftIIrRqj4Lne3/5kRvLLYI+YjmuXuHXZSA/0l5DBcT\nKBFGyrcdUprdWx42GbeBuxpH5cHOzLlUeqmFkqBHgnpAwtdV4+F3Lko2s3MCAwEA\nAQKCAgAJAtkhFs9j2lRaJiAlCAdqsSf7FhHSQshkJFbwAspBGJx8tJkkRYf34SHP\nq2HxXDkmGQZz6HkA6EG6jrO7Auo1D2CFrZBTFg2O1CTb8I36xO7J9FY2e4FE0U4i\n9WnxYnDqaugy6G4aPmzhzLd8+xeOpA1wvY4/rNT8uKFG2hEbNVzwOBdlVtEUZOKK\n4P1TT9FBU85JyrmK4QiLMK6KMG6ElnjCzbybvGUuctj0m6WPuLl0AgfxMRXNIawa\nEWu96/3cRt1c1W5cGW4RrMmP7MpdVhc+SOKWpWM52grE0Gn7+lVMK/j+rNojlaKS\ntOz6T7Tv6ZkHvPyeYLvwF2b7EHEQlzWNGZAa5U1J9vtaqGvYBY0V3SrNzaj2yll0\n1uJ0EQGOB++87AN+5HYxlrWMsQBnGUMw7aSMzkSwd31otLStWtt8BtWBiy4l0GCq\ny/jfJ4S3n36vExg+QztqSq1AyoQGTgNBd/djCnxb43A9ZWXuO4SpTFYKeIbp+5wy\naBDHnqH3t2J9/jCS/pfQmVetwTS/Ah5RkZiBcMV86alVgvoYMINDRWdO+Qgkgotj\niSyiHGAqrWU+jpGxPGgWe5cvKEG+JEkUBgTDoq4lAvYz7rcETTezy4vsHesaa0xl\nLd9uEq8e+rQ+ykjpgCtTV/9Bz639MpCSndgJ5FdvEoPXRwTzYQKCAQEAy6EYRV8w\njQ25BSOQU87t8XHMFwkVGAKEj6x4yo3xMBzdtj1Onn3I8N5QUGMxsc1S+gDW9ff8\nal5aHKFOR05Mx1CvksQ+3XmFxSeTjP5/UdoY2TvG1p+eYcDn7jcSSuID/juBYiO1\n9T3N1bf32w4PoUxxcu8CbGEgfi3PSlNH4NvvNOfAv3315J5bsBT1tKWMJZpt26xM\njDnALZnOoM+c5T5SCaq4n70JXXbElgeDJ1CRlc3b7vXm5VuXa34OVjSrQMfaycDI\nvHuN3439cwqlv1yVVlr7VvLHK1+GSbrTCPFklxizHW0rfaOB5t8DcWsTHjFK7/26\nvVo7kxgzlBm9oQKCAQEAwLOWZydYXQ+IsLfQUfwX3HYiTUrLNuIbsdVGhLlh46ri\nFEVlmnpvX6X31s1hlVxWnwqePItrcqc+6UZpsiOtm4TaAcKcy91Ojxw4DYaSVKMe\nC1u8JRGT+8LUYGwwwy9P+Cjs4BLc/M0gBYwVtwcsnVBhc9aBgV57UnrLEI+qSWh9\ncStAGqawAlyPerJovlHflumKY0XTcWEEyBjFGycERzOei94s0O+oMq6LL7r2Pard\nFYIw5qw44CppWs8GjgtStgJweFEpVpf/I8L2406GtM00qDhbj0ppjlKrMtnd7tm7\nX3kzJDhwmJC4F0nsitn0js3V7WuMFFueySqZ/9zQkwKCAQA3BTQ1ql7H/ik8Bjv9\nKTCJyKHIp5+XeTawS33n3SqKQYJ2t2IC2SHuU2OIFQVccYo2Gvv6QiNl6lCW487u\n3W1JfbezBGuoFCnc97CarDliuHd2WvyDCWYWK7V/XEoJCcDidNmk1tlXhneTg73o\n5oe6oZkV4RbaifPppB8aih4l18dumCAwnaxPNaq4jTFGumR/1JZSgN++E+ylZaW7\nNd3C42/vRL3WC5H4OxsrHqSFfvOXcqQ2kSLuXi0v0kO1b03Glzm6weh7ZTcUbJ1K\n9FmMt0typeq7t6WxRoJWcbCL/g2b9gSPsuvJauCyrReVEcV8pv0Ek55dg4ANYsCt\nB7eBAoIBAA1Ml6W43m1VxB5GIVd3KYJqmMVy0wGPq0+jo4FHdg0LXxz9CR0XNBaK\n0sb9V63COKwzRGZzTNAV4M6isGq98Z5ZFd6nOsELHf/cd7XfQrwaodkw1JROWUM3\n2AZrkRX3bvJ2iJqSUrQTyBD58Qm3sgmNYZxoAnoWNjakCN20vJDiaD3mOT9dn6oa\nD0DtjTMNXXwB1C4zxHr/sh/JLdlI4Q+07U1D4R3uXFJbH3oE0jfBUg7I9G0HF8nI\nw4qj3RTyjRvIoVB8cXjrdo1WSEm5wqw5uVeexBrojM5owC37QUjZuAfHypLGXmb5\nHetli7CtiCZO6GPBV2KUTCogCY7SA4cCggEAa47g/k0gP/z3DgnoeRs7fCmsJgDO\nX3FRKZAM4trdzhy+XXflOQ9pMFaGywnRY4qyb29f6/NVUv0tck79Z/fDadi8gqYO\nM7yEnt25xWtVHHQPyBtgnX8VxRFs8OmzzEZ/b8tXMm++RLyqe4FKZE5FK5ukzqwU\nuN6qiS3g1SUongfx4uoIaXRPmfvHz8ooqhX802Io4gj3y1D8yVszS4oWz6w3w/Fj\ndixGYhzWwudLBiaiQW5sXEOaANuS9PaeaZzA0abSXXLIyQ9ASgAts16CNCgkKTie\nZh6VWe/8fpyQGDgY59VYo9wLIjtOTZ9jrYfjSOwnl4BVg5GlAULaNTllYw==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-13T10:27:16.717 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-13T10:27:17.334 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:17 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:17.334 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:17 smithi060 ceph-mon[27375]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:17.335 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:17 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:18.009 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-13T10:27:18.009 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial admin user... 2024-09-13T10:27:18.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:18 smithi060 ceph-mon[27375]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:18.585 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:19.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:19 smithi060 ceph-mon[27375]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:19.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:19 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:19.854 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$W0biCNL4OtPJj/jsiLzB7evq2raQjEngPc/ffTy0Dqi0vhi8JVYYy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726223238, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-13T10:27:19.854 INFO:teuthology.orchestra.run.smithi060.stdout:Fetching dashboard port number... 2024-09-13T10:27:20.797 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:20 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3128165121' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-13T10:27:21.059 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 8443 2024-09-13T10:27:21.070 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T10:27:21.070 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-13T10:27:21.078 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T10:27:21.078 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-13T10:27:21.078 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-13T10:27:21.078 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-13T10:27:21.080 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph Dashboard is now available at: 2024-09-13T10:27:21.080 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:21.080 INFO:teuthology.orchestra.run.smithi060.stdout: URL: https://smithi060.front.sepia.ceph.com:8443/ 2024-09-13T10:27:21.081 INFO:teuthology.orchestra.run.smithi060.stdout: User: admin 2024-09-13T10:27:21.081 INFO:teuthology.orchestra.run.smithi060.stdout: Password: 1uq1xnmyko 2024-09-13T10:27:21.081 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:21.081 INFO:teuthology.orchestra.run.smithi060.stdout:Saving cluster configuration to /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config directory 2024-09-13T10:27:22.416 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-13T10:27:22.416 INFO:teuthology.orchestra.run.smithi060.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-13T10:27:22.416 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.416 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout:Or, if you are only running a single cluster on this host: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: ceph telemetry on 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout:For more information see: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:22.417 INFO:teuthology.orchestra.run.smithi060.stdout:Bootstrap complete. 2024-09-13T10:27:22.465 INFO:tasks.cephadm:Fetching config... 2024-09-13T10:27:22.465 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:27:22.465 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-13T10:27:22.482 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-13T10:27:22.482 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:27:22.482 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-13T10:27:22.538 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-13T10:27:22.538 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:27:22.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-13T10:27:22.603 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-13T10:27:22.603 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:27:22.603 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-13T10:27:22.658 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-13T10:27:22.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T10:27:22.735 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:27:22.749 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T10:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:27:22.882 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T10:27:22.944 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpNI+rRm+Gfpc+X+YUmtkIskMQxJYrwNUrQ/60KyUhbo1Bo8QM3rOFmWMm0DWsJ7duVtkpVy9DRIcuS5HCZ4xUU9Ozj7lS8a7CZ3cW4ZpjHLryABizKfXv8zY8kcqU/SRP55+ThuC/h69FFguR/bDkXQ5nNYcp4+fM3DzQXoECc0KjG1ggn1RsAZyHn1ksQGZeQFQjJEXFYTW6ucKGsVvdhBhOiI7Zbt6ncaegn7ZDn1ZmzbJjCQwouER1NOGW+LTBpvGmXG4CIpgu5XOYKmojTQNM+dXDLFwO/2+bVj9ISENQ6xZs0h30UbCm6yDKIcp/mU/ecGG6zDJnSwqxjSlZujArLyXil0Frzw+DcFghshkP32p9a8NPznCGHm0lt20iY1nVgXqoalYbF4snltKdDSZ3E4fwLPJjZ3wz20GYr9w+rCIe9rbkoht9CuuTjl+42z77yvImxmhpHK5LHs9mRSnUY45LoWQhJtr8XVfZExnJmoZ04z/f5prA6tO9NIk= ceph-74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:27:22.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-13T10:27:22.975 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:22 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/312158500' entity='client.admin' 2024-09-13T10:27:23.215 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:24.882 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-13T10:27:24.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-13T10:27:25.148 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2138316177' entity='client.admin' 2024-09-13T10:27:25.148 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: mgrmap e14: y(active, since 12s) 2024-09-13T10:27:25.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:25.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:25.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"} : dispatch 2024-09-13T10:27:25.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:25.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:25.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:26.822 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-09-13T10:27:26.822 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:27:26.822 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-09-13T10:27:26.842 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:27:26.843 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:26.901 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-09-13T10:27:26.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch host add smithi138 2024-09-13T10:27:27.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:27.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:27.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:27.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:27.270 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:27.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-13T10:27:27.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: Updating smithi060:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:27:27.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:27.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:27.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:27.271 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:27.337 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:28.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:28 smithi060 ceph-mon[27375]: Updating smithi060:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T10:27:29.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:29 smithi060 ceph-mon[27375]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:30.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:30 smithi060 ceph-mon[27375]: Deploying cephadm binary to smithi138 2024-09-13T10:27:31.828 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-09-13T10:27:32.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T10:27:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:33.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:32 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:33.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:32 smithi060 ceph-mon[27375]: Added host smithi138 2024-09-13T10:27:33.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:32 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:33.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:32 smithi060 ceph-mon[27375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:33.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:32 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:33.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:32 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:33.372 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-09-13T10:27:34.044 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-09-13T10:27:34.045 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:27:34.045 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-09-13T10:27:34.065 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:27:34.065 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:34.124 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-09-13T10:27:34.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch host add smithi173 2024-09-13T10:27:34.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:34.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:34 smithi060 ceph-mon[27375]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:27:34.374 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:35.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:35 smithi060 ceph-mon[27375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:36.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:37.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: Updating smithi138:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: Deploying cephadm binary to smithi173 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: Updating smithi138:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:37.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:38.889 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-09-13T10:27:39.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:39 smithi060 ceph-mon[27375]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:39.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:39.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:39.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:39.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T10:27:39.725 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:40.562 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:27:40.563 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-09-13T10:27:40.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:40 smithi060 ceph-mon[27375]: Added host smithi173 2024-09-13T10:27:40.862 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:40 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:40.863 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:40 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:41.128 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-13T10:27:41.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-13T10:27:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:27:41.740 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:41 smithi060 ceph-mon[27375]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:41.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:41 smithi060 ceph-mon[27375]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:27:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-09-13T10:27:42.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3484257998' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-13T10:27:43.021 INFO:tasks.cephadm:Adding mon.a on smithi060 2024-09-13T10:27:43.021 INFO:tasks.cephadm:Adding mon.b on smithi138 2024-09-13T10:27:43.021 INFO:tasks.cephadm:Adding mon.c on smithi173 2024-09-13T10:27:43.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c' 2024-09-13T10:27:43.275 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-13T10:27:43.321 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-13T10:27:43.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:43.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3484257998' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"} : dispatch 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:44.030 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2024-09-13T10:27:44.627 DEBUG:teuthology.orchestra.run.smithi138:mon.b> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.b.service 2024-09-13T10:27:44.630 DEBUG:teuthology.orchestra.run.smithi173:mon.c> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.c.service 2024-09-13T10:27:44.633 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T10:27:44.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:27:44.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:44.905 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:27:45.550 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:27:45.550 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","modified":"2024-09-13T10:26:21.083088Z","created":"2024-09-13T10:26:21.083088Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-13T10:27:45.550 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-09-13T10:27:45.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:45 smithi060 ceph-mon[27375]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T10:27:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:45 smithi060 ceph-mon[27375]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:45 smithi060 ceph-mon[27375]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:45 smithi060 ceph-mon[27375]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c;count:3 2024-09-13T10:27:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:45 smithi060 ceph-mon[27375]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T10:27:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:45 smithi060 ceph-mon[27375]: Reconfiguring daemon mon.a on smithi060 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: Deploying daemon mon.c on smithi173 2024-09-13T10:27:46.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:46 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/73114197' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:47.183 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T10:27:47.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T10:27:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:47 smithi060 ceph-mon[27375]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:48.045 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:27:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:27:48.906 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","modified":"2024-09-13T10:26:21.083088Z","created":"2024-09-13T10:26:21.083088Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-13T10:27:48.907 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-09-13T10:27:49.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:49 smithi060 ceph-mon[27375]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:49.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:49 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/2218821857' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: rocksdb: (Original Log Time 2024/09/13-10:27:49.891783) [db/db_impl/db_impl_compaction_flush.cc:2832] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: rocksdb: [db/flush_job.cc:856] [default] [JOB 3] Flushing memtable with next log file: 13 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726223269891895, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 1091, "num_deletes": 13, "total_data_size": 2551488, "memory_usage": 2667936, "flush_reason": "Manual Compaction"} 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: rocksdb: [db/flush_job.cc:885] [default] [JOB 3] Level-0 flush table #14: started 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).mds e1 new map 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout: e1 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout: btime 2024-09-13T10:26:24:281765+0000 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T10:27:49.895 INFO:journalctl@ceph.mon.c.smithi173.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-13T10:27:49.896 INFO:journalctl@ceph.mon.c.smithi173.stdout: legacy client fscid: -1 2024-09-13T10:27:49.896 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-09-13T10:27:49.896 INFO:journalctl@ceph.mon.c.smithi173.stdout: No filesystems configured 2024-09-13T10:27:49.896 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:50.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Added host smithi173 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3484257998' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3484257998' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:27:50.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"} : dispatch 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.285 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:50.286 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c;count:3 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Reconfiguring daemon mon.a on smithi060 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:50.287 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:50.288 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: Deploying daemon mon.c on smithi173 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/73114197' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/2218821857' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 ceph-mon[30864]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 bash[30845]: 034d4332344d4f844d1132e6c2bc6fae2ce3f13bb63f7449cf0f4a4e66ce8d2c 2024-09-13T10:27:50.289 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:49 smithi173 systemd[1]: Started Ceph mon.c for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:27:50.401 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T10:27:50.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T10:27:50.661 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:27:55.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: mon.a calling monitor election 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: mon.c calling monitor election 2024-09-13T10:27:55.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: monmap epoch 2 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: last_changed 2024-09-13T10:27:49.898786+0000 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: min_mon_release 19 (squid) 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: election_strategy: 1 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:27:55.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: fsmap 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: mgrmap e14: y(active, since 43s) 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: overall HEALTH_OK 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:55.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: mon.a calling monitor election 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: mon.c calling monitor election 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: monmap epoch 2 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: last_changed 2024-09-13T10:27:49.898786+0000 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: min_mon_release 19 (squid) 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: election_strategy: 1 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:27:55.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: fsmap 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: mgrmap e14: y(active, since 43s) 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: overall HEALTH_OK 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:55.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:55.766 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:27:55.767 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","modified":"2024-09-13T10:27:49.898786Z","created":"2024-09-13T10:26:21.083088Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T10:27:55.767 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-09-13T10:27:56.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:55 smithi173 ceph-mon[30864]: Deploying daemon mon.b on smithi138 2024-09-13T10:27:56.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:55 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/533980663' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:56.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:55 smithi060 ceph-mon[27375]: Deploying daemon mon.b on smithi138 2024-09-13T10:27:56.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:55 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/533980663' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:57.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:56 smithi173 ceph-mon[30864]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:57.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:27:56 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:57.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:56 smithi060 ceph-mon[27375]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:57.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:27:56 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:57.399 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T10:27:57.399 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T10:27:57.651 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:27:58.531 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:27:58.532 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","modified":"2024-09-13T10:27:49.898786Z","created":"2024-09-13T10:26:21.083088Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T10:27:58.532 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-09-13T10:27:58.581 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: rocksdb: [db/db_impl/db_impl_write.cc:2098] [default] New memtable created with log file: #13. Immutable memtables: 0. 2024-09-13T10:27:58.581 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: rocksdb: (Original Log Time 2024/09/13-10:27:58.579261) [db/db_impl/db_impl_compaction_flush.cc:2832] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-09-13T10:27:58.581 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: rocksdb: [db/flush_job.cc:856] [default] [JOB 3] Flushing memtable with next log file: 13 2024-09-13T10:27:58.581 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726223278579383, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 1163, "num_deletes": 13, "total_data_size": 2606762, "memory_usage": 2724368, "flush_reason": "Manual Compaction"} 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: rocksdb: [db/flush_job.cc:885] [default] [JOB 3] Level-0 flush table #14: started 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).mds e1 new map 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: e1 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: btime 2024-09-13T10:26:24:281765+0000 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: legacy client fscid: -1 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout: No filesystems configured 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-13T10:27:58.582 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-13T10:27:58.583 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-13T10:27:58.583 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-13T10:27:58.583 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-13T10:27:58.583 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Added host smithi173 2024-09-13T10:27:58.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3484257998' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3484257998' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"} : dispatch 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi138:172.21.15.138=b;smithi173:172.21.15.173=c;count:3 2024-09-13T10:27:58.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Reconfiguring daemon mon.a on smithi060 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Deploying daemon mon.c on smithi173 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/73114197' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/2218821857' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.a calling monitor election 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.c calling monitor election 2024-09-13T10:27:58.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: monmap epoch 2 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: last_changed 2024-09-13T10:27:49.898786+0000 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: min_mon_release 19 (squid) 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: election_strategy: 1 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:27:58.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: fsmap 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mgrmap e14: y(active, since 43s) 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: overall HEALTH_OK 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: Deploying daemon mon.b on smithi138 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/533980663' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 ceph-mon[31033]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-13T10:27:58.858 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 bash[31014]: 51693fe7572c4caf5f1fcffc7ead30c4413d50a27143fb8ad731a484d49092e4 2024-09-13T10:27:58.859 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:27:58 smithi138 systemd[1]: Started Ceph mon.b for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:28:00.112 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T10:28:00.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T10:28:00.366 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:28:03.945 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: mon.c calling monitor election 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: mon.a calling monitor election 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:03.946 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: monmap epoch 3 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: last_changed 2024-09-13T10:27:58.584911+0000 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: min_mon_release 19 (squid) 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: election_strategy: 1 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-13T10:28:03.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: fsmap 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: mgrmap e14: y(active, since 51s) 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: mon.b (rank 2) addr [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] is down (out of quorum) 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:03.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: mon.c calling monitor election 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: mon.a calling monitor election 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: monmap epoch 3 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: last_changed 2024-09-13T10:27:58.584911+0000 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: min_mon_release 19 (squid) 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: election_strategy: 1 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-13T10:28:04.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: fsmap 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: mgrmap e14: y(active, since 51s) 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: mon.b (rank 2) addr [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] is down (out of quorum) 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:04.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:04.049 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:28:04.049 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":3,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","modified":"2024-09-13T10:27:58.584911Z","created":"2024-09-13T10:26:21.083088Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T10:28:04.049 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 3 2024-09-13T10:28:04.636 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-13T10:28:04.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-13T10:28:04.879 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:04.879 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:04 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/2916016114' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:28:04.879 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:28:04.947 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:04.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:04 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/2916016114' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:28:04.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.597 INFO:teuthology.orchestra.run.smithi060.stdout:# minimal ceph.conf for 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:05.597 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-09-13T10:28:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: mon_host = [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-09-13T10:28:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: mon.c calling monitor election 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: mon.a calling monitor election 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: monmap epoch 3 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: last_changed 2024-09-13T10:27:58.584911+0000 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: min_mon_release 19 (squid) 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: election_strategy: 1 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: fsmap 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: mgrmap e14: y(active, since 51s) 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T10:28:05.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: mon.b (rank 2) addr [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] is down (out of quorum) 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/2916016114' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:28:05.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:06.293 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-13T10:28:06.294 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:28:06.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T10:28:06.322 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:28:06.322 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:28:06.389 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:28:06.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: mon.b calling monitor election 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Updating smithi138:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Updating smithi060:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: mon.b calling monitor election 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: mon.c calling monitor election 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: mon.a calling monitor election 2024-09-13T10:28:06.413 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: monmap epoch 3 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: last_changed 2024-09-13T10:27:58.584911+0000 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: min_mon_release 19 (squid) 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: election_strategy: 1 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: fsmap 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: mgrmap e14: y(active, since 53s) 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T10:28:06.414 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: Cluster is now healthy 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: overall HEALTH_OK 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:06.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:06.417 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:28:06.417 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:28:06.482 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:28:06.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T10:28:06.510 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:28:06.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T10:28:06.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: mon.b calling monitor election 2024-09-13T10:28:06.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-13T10:28:06.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-13T10:28:06.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-13T10:28:06.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Updating smithi138:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Updating smithi060:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: mon.b calling monitor election 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: mon.c calling monitor election 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: mon.a calling monitor election 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: monmap epoch 3 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: last_changed 2024-09-13T10:27:58.584911+0000 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: min_mon_release 19 (squid) 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: election_strategy: 1 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:28:06.569 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: fsmap 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: mgrmap e14: y(active, since 53s) 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: Cluster is now healthy 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: overall HEALTH_OK 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.570 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:06.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:06.575 INFO:tasks.cephadm:Adding mgr.y on smithi060 2024-09-13T10:28:06.575 INFO:tasks.cephadm:Adding mgr.x on smithi138 2024-09-13T10:28:06.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi060=y;smithi138=x' 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: mon.b calling monitor election 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Updating smithi173:/etc/ceph/ceph.conf 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Updating smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Updating smithi138:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.637 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Updating smithi060:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: mon.b calling monitor election 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: mon.c calling monitor election 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: mon.a calling monitor election 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: monmap epoch 3 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: last_changed 2024-09-13T10:27:58.584911+0000 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: created 2024-09-13T10:26:21.083088+0000 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: min_mon_release 19 (squid) 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: election_strategy: 1 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-13T10:28:06.638 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: fsmap 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: mgrmap e14: y(active, since 53s) 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: Cluster is now healthy 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: overall HEALTH_OK 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:06.639 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:06.640 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:06.849 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:28:07.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:07 smithi060 ceph-mon[27375]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:07.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:07 smithi060 ceph-mon[27375]: Reconfiguring mon.a (monmap changed)... 2024-09-13T10:28:07.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:07 smithi060 ceph-mon[27375]: Reconfiguring daemon mon.a on smithi060 2024-09-13T10:28:07.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:07 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:07.479 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mgr update... 2024-09-13T10:28:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:07 smithi173 ceph-mon[30864]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:07 smithi173 ceph-mon[30864]: Reconfiguring mon.a (monmap changed)... 2024-09-13T10:28:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:07 smithi173 ceph-mon[30864]: Reconfiguring daemon mon.a on smithi060 2024-09-13T10:28:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:07 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:07 smithi138 ceph-mon[31033]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:07 smithi138 ceph-mon[31033]: Reconfiguring mon.a (monmap changed)... 2024-09-13T10:28:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:07 smithi138 ceph-mon[31033]: Reconfiguring daemon mon.a on smithi060 2024-09-13T10:28:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:07 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:08.082 DEBUG:teuthology.orchestra.run.smithi138:mgr.x> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.x.service 2024-09-13T10:28:08.085 DEBUG:tasks.cephadm:set 0 configs 2024-09-13T10:28:08.085 INFO:tasks.cephadm:Deploying OSDs... 2024-09-13T10:28:08.085 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:28:08.085 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:28:08.108 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:28:08.108 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-09-13T10:28:08.168 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:28:08.168 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:08.168 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-13T10:28:08.168 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:08.169 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:08.169 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 10:27:25.690979073 +0000 2024-09-13T10:28:08.169 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 10:23:10.898149161 +0000 2024-09-13T10:28:08.169 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 10:23:10.898149161 +0000 2024-09-13T10:28:08.169 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-13 10:23:10.898149161 +0000 2024-09-13T10:28:08.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:28:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T10:28:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T10:28:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000117245 s, 4.4 MB/s 2024-09-13T10:28:08.239 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:28:08.299 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-09-13T10:28:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:28:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:08.358 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T10:28:08.359 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:08.359 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:08.359 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 10:27:25.726978730 +0000 2024-09-13T10:28:08.359 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 10:23:11.229145975 +0000 2024-09-13T10:28:08.359 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 10:23:11.229145975 +0000 2024-09-13T10:28:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-13 10:23:11.229145975 +0000 2024-09-13T10:28:08.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:28:08.418 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: Reconfiguring mon.b (monmap changed)... 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: Reconfiguring daemon mon.b on smithi138 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi060=y;smithi138=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: Saving service mgr spec with placement smithi060=y;smithi138=x;count:2 2024-09-13T10:28:08.419 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T10:28:08.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T10:28:08.426 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000122644 s, 4.2 MB/s 2024-09-13T10:28:08.427 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:28:08.486 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-09-13T10:28:08.543 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:28:08.544 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:08.544 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-13T10:28:08.544 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:08.544 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:08.545 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 10:27:25.761978396 +0000 2024-09-13T10:28:08.545 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 10:23:11.547142914 +0000 2024-09-13T10:28:08.545 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 10:23:11.547142914 +0000 2024-09-13T10:28:08.545 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-13 10:23:11.547142914 +0000 2024-09-13T10:28:08.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:28:08.608 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.608 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.609 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.609 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: Reconfiguring mon.b (monmap changed)... 2024-09-13T10:28:08.609 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:08.609 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:08.609 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:08.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: Reconfiguring daemon mon.b on smithi138 2024-09-13T10:28:08.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi060=y;smithi138=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:08.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: Saving service mgr spec with placement smithi060=y;smithi138=x;count:2 2024-09-13T10:28:08.610 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T10:28:08.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T10:28:08.613 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000111623 s, 4.6 MB/s 2024-09-13T10:28:08.614 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:28:08.671 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-09-13T10:28:08.728 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 10:27:25.797978053 +0000 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 10:23:11.876139748 +0000 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 10:23:11.876139748 +0000 2024-09-13T10:28:08.729 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-13 10:23:11.876139748 +0000 2024-09-13T10:28:08.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: Reconfiguring mon.b (monmap changed)... 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: Reconfiguring daemon mon.b on smithi138 2024-09-13T10:28:08.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi060=y;smithi138=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:08.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: Saving service mgr spec with placement smithi060=y;smithi138=x;count:2 2024-09-13T10:28:08.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T10:28:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T10:28:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000111849 s, 4.6 MB/s 2024-09-13T10:28:08.795 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:28:08.852 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:28:08.853 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:28:08.869 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:28:08.869 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-09-13T10:28:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-13 10:27:58.020020404 +0000 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-13 10:23:05.251788591 +0000 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-13 10:23:05.251788591 +0000 2024-09-13T10:28:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-13 10:23:05.251788591 +0000 2024-09-13T10:28:08.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:28:08.988 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-13T10:28:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-13T10:28:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000191275 s, 2.7 MB/s 2024-09-13T10:28:08.990 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:28:09.045 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-13 10:27:58.020020404 +0000 2024-09-13T10:28:09.101 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-13 10:23:05.558781682 +0000 2024-09-13T10:28:09.102 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-13 10:23:05.558781682 +0000 2024-09-13T10:28:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-13 10:23:05.558781682 +0000 2024-09-13T10:28:09.102 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:28:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-13T10:28:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-13T10:28:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000204101 s, 2.5 MB/s 2024-09-13T10:28:09.166 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:28:09.222 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-13 10:27:58.021020382 +0000 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-13 10:23:05.880774436 +0000 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-13 10:23:05.880774436 +0000 2024-09-13T10:28:09.278 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-13 10:23:05.880774436 +0000 2024-09-13T10:28:09.279 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:28:09.341 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-13T10:28:09.341 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-13T10:28:09.342 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000211014 s, 2.4 MB/s 2024-09-13T10:28:09.343 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:28:09.401 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-09-13T10:28:09.458 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:28:09.458 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:09.458 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T10:28:09.458 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:09.458 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:09.459 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-13 10:27:58.021020382 +0000 2024-09-13T10:28:09.459 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-13 10:23:06.188767504 +0000 2024-09-13T10:28:09.459 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-13 10:23:06.188767504 +0000 2024-09-13T10:28:09.459 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-13 10:23:06.188767504 +0000 2024-09-13T10:28:09.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:28:09.522 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-13T10:28:09.522 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-13T10:28:09.522 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000217686 s, 2.4 MB/s 2024-09-13T10:28:09.523 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:28:09.580 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:28:09.580 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:28:09.597 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:28:09.597 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-09-13T10:28:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:09 smithi138 ceph-mon[31033]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:09.652 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:28:09.652 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:09.652 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-13T10:28:09.653 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:09.653 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:09.653 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-13 10:27:48.999542674 +0000 2024-09-13T10:28:09.653 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-13 10:23:06.574951271 +0000 2024-09-13T10:28:09.653 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-13 10:23:06.574951271 +0000 2024-09-13T10:28:09.653 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-13 10:23:06.574951271 +0000 2024-09-13T10:28:09.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:28:09.714 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:09 smithi173 ceph-mon[30864]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:09.714 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:09.714 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:09.715 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:09.715 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:09.715 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:09.717 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-13T10:28:09.717 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-13T10:28:09.717 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000343185 s, 1.5 MB/s 2024-09-13T10:28:09.719 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:28:09.775 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-09-13T10:28:09.830 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:28:09.830 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:09.830 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T10:28:09.830 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:09.831 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:09.831 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-13 10:27:48.999542674 +0000 2024-09-13T10:28:09.831 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-13 10:23:06.920943571 +0000 2024-09-13T10:28:09.831 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-13 10:23:06.920943571 +0000 2024-09-13T10:28:09.831 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-13 10:23:06.920943571 +0000 2024-09-13T10:28:09.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:28:09.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:09 smithi060 ceph-mon[27375]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:09.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:09.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:09.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-13T10:28:09.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-13T10:28:09.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:09.894 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-13T10:28:09.894 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-13T10:28:09.894 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000200616 s, 2.6 MB/s 2024-09-13T10:28:09.895 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:28:09.952 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-09-13T10:28:10.008 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:28:10.008 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-13 10:27:48.999542674 +0000 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-13 10:23:07.245936338 +0000 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-13 10:23:07.245936338 +0000 2024-09-13T10:28:10.009 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-13 10:23:07.245936338 +0000 2024-09-13T10:28:10.009 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:28:10.073 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-13T10:28:10.073 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-13T10:28:10.073 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.00017699 s, 2.9 MB/s 2024-09-13T10:28:10.075 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:28:10.131 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-09-13T10:28:10.186 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:28:10.186 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:28:10.186 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-13T10:28:10.187 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:28:10.187 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T10:28:10.187 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-13 10:27:49.000542652 +0000 2024-09-13T10:28:10.187 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-13 10:23:07.571929083 +0000 2024-09-13T10:28:10.187 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-13 10:23:07.571929083 +0000 2024-09-13T10:28:10.187 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-13 10:23:07.571929083 +0000 2024-09-13T10:28:10.187 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:28:10.250 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-13T10:28:10.250 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-13T10:28:10.250 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000271438 s, 1.9 MB/s 2024-09-13T10:28:10.251 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:28:10.308 INFO:tasks.cephadm:Deploying osd.0 on smithi060 with /dev/vg_nvme/lv_4... 2024-09-13T10:28:10.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-13T10:28:10.571 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:28:10.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: Reconfiguring mon.c (monmap changed)... 2024-09-13T10:28:10.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: Reconfiguring daemon mon.c on smithi173 2024-09-13T10:28:10.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T10:28:10.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:10.772 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: Reconfiguring mon.c (monmap changed)... 2024-09-13T10:28:10.772 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: Reconfiguring daemon mon.c on smithi173 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T10:28:10.773 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: Reconfiguring mon.c (monmap changed)... 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: Reconfiguring daemon mon.c on smithi173 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:10.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:10.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T10:28:10.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T10:28:10.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T10:28:10.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:11.722 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:11 smithi060 ceph-mon[27375]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:11.722 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:11 smithi060 ceph-mon[27375]: Deploying daemon mgr.x on smithi138 2024-09-13T10:28:11.755 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:11 smithi138 ceph-mon[31033]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:11.755 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:11 smithi138 ceph-mon[31033]: Deploying daemon mgr.x on smithi138 2024-09-13T10:28:11.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:11 smithi173 ceph-mon[30864]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:11.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:11 smithi173 ceph-mon[30864]: Deploying daemon mgr.x on smithi138 2024-09-13T10:28:12.100 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:11 smithi138 systemd[1]: Starting Ceph mgr.x for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:28:12.421 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 podman[32385]: 2024-09-13 10:28:12.038239844 +0000 UTC m=+0.015708951 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:28:12.422 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 podman[32385]: 2024-09-13 10:28:12.143277814 +0000 UTC m=+0.120746915 container create 826cd498145f68df66c3d6ec9f51383defd70192e574da28832c4ab9ac6a5b90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-13T10:28:12.422 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 podman[32385]: 2024-09-13 10:28:12.33707485 +0000 UTC m=+0.314543965 container init 826cd498145f68df66c3d6ec9f51383defd70192e574da28832c4ab9ac6a5b90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-13T10:28:12.422 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 podman[32385]: 2024-09-13 10:28:12.339849392 +0000 UTC m=+0.317318500 container start 826cd498145f68df66c3d6ec9f51383defd70192e574da28832c4ab9ac6a5b90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-13T10:28:12.422 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 ceph-mgr[32404]: -- 172.21.15.138:0/1339225757 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617ea8e90e0 con 0x5617ea8c7400 2024-09-13T10:28:12.422 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 bash[32385]: 826cd498145f68df66c3d6ec9f51383defd70192e574da28832c4ab9ac6a5b90 2024-09-13T10:28:12.850 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:28:12 smithi138 systemd[1]: Started Ceph mgr.x for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:28:13.083 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:28:13.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2024-09-13T10:28:13.362 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:28:13.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:13 smithi060 ceph-mon[27375]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:13.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:13 smithi173 ceph-mon[30864]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:13 smithi138 ceph-mon[31033]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:14.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:14.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:14.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:14.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:14.829 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:14.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T10:28:14.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T10:28:14.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:14.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:28:14.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:28:14.830 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:28:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:15.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:15.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:28:15.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:15.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:15 smithi060 ceph-mon[27375]: Reconfiguring mgr.y (unknown last config time)... 2024-09-13T10:28:15.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:15 smithi060 ceph-mon[27375]: Reconfiguring daemon mgr.y on smithi060 2024-09-13T10:28:15.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:15 smithi060 ceph-mon[27375]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:15.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:15 smithi060 ceph-mon[27375]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:15 smithi173 ceph-mon[30864]: Reconfiguring mgr.y (unknown last config time)... 2024-09-13T10:28:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:15 smithi173 ceph-mon[30864]: Reconfiguring daemon mgr.y on smithi060 2024-09-13T10:28:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:15 smithi173 ceph-mon[30864]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:15 smithi173 ceph-mon[30864]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:15 smithi138 ceph-mon[31033]: Reconfiguring mgr.y (unknown last config time)... 2024-09-13T10:28:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:15 smithi138 ceph-mon[31033]: Reconfiguring daemon mgr.y on smithi060 2024-09-13T10:28:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:15 smithi138 ceph-mon[31033]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:15 smithi138 ceph-mon[31033]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:17.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:28:17.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:17.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:17.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:17.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:18.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3026244242' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd"} : dispatch 2024-09-13T10:28:18.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3026244242' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd"}]': finished 2024-09-13T10:28:18.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: Standby manager daemon x started 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T10:28:18.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:17 smithi138 ceph-mon[31033]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T10:28:18.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3026244242' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd"} : dispatch 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3026244242' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd"}]': finished 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: Standby manager daemon x started 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T10:28:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:17 smithi060 ceph-mon[27375]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T10:28:18.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3026244242' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd"} : dispatch 2024-09-13T10:28:18.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3026244242' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd"}]': finished 2024-09-13T10:28:18.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T10:28:18.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:18.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:18.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: Standby manager daemon x started 2024-09-13T10:28:18.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-13T10:28:18.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-13T10:28:18.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-13T10:28:18.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:17 smithi173 ceph-mon[30864]: from='mgr.? 172.21.15.138:0/2582153509' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-13T10:28:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:18 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/215815497' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:28:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:18 smithi138 ceph-mon[31033]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:18 smithi138 ceph-mon[31033]: mgrmap e15: y(active, since 66s), standbys: x 2024-09-13T10:28:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T10:28:19.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:18 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/215815497' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:28:19.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:18 smithi060 ceph-mon[27375]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:19.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:18 smithi060 ceph-mon[27375]: mgrmap e15: y(active, since 66s), standbys: x 2024-09-13T10:28:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T10:28:19.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:18 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/215815497' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:28:19.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:18 smithi173 ceph-mon[30864]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:19.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:18 smithi173 ceph-mon[30864]: mgrmap e15: y(active, since 66s), standbys: x 2024-09-13T10:28:19.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-13T10:28:21.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:20 smithi060 ceph-mon[27375]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:21.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:20 smithi173 ceph-mon[30864]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:21.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:20 smithi138 ceph-mon[31033]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:23.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:22 smithi060 ceph-mon[27375]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:23.308 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:22 smithi173 ceph-mon[30864]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:23.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:22 smithi138 ceph-mon[31033]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:25.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:24 smithi060 ceph-mon[27375]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:25.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-13T10:28:25.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:25.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:24 smithi060 ceph-mon[27375]: Deploying daemon osd.0 on smithi060 2024-09-13T10:28:25.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:24 smithi173 ceph-mon[30864]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:25.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-13T10:28:25.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:25.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:24 smithi173 ceph-mon[30864]: Deploying daemon osd.0 on smithi060 2024-09-13T10:28:25.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:24 smithi138 ceph-mon[31033]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:25.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-13T10:28:25.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:25.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:24 smithi138 ceph-mon[31033]: Deploying daemon osd.0 on smithi060 2024-09-13T10:28:27.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:26 smithi060 ceph-mon[27375]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:27.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:26 smithi173 ceph-mon[30864]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:27.340 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:26 smithi138 ceph-mon[31033]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:29.015 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:28 smithi060 ceph-mon[27375]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:29.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:28 smithi173 ceph-mon[30864]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:29.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:28 smithi138 ceph-mon[31033]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:30.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:30.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:30.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:30.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:30.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:30.304 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:30.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:30.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:30.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:31 smithi060 ceph-mon[27375]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:31 smithi173 ceph-mon[30864]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:32.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:31 smithi138 ceph-mon[31033]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:33 smithi173 ceph-mon[30864]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:33.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:33 smithi138 ceph-mon[31033]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:33 smithi060 ceph-mon[27375]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.106 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 0 on host 'smithi060' 2024-09-13T10:28:35.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:35 smithi060 ceph-mon[27375]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:35.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:35 smithi060 ceph-mon[27375]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T10:28:35.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:35.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:35 smithi173 ceph-mon[30864]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:35 smithi173 ceph-mon[30864]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T10:28:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:35 smithi138 ceph-mon[31033]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:35 smithi138 ceph-mon[31033]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-13T10:28:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:35.725 DEBUG:teuthology.orchestra.run.smithi060:osd.0> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.0.service 2024-09-13T10:28:35.728 INFO:tasks.cephadm:Deploying osd.1 on smithi060 with /dev/vg_nvme/lv_3... 2024-09-13T10:28:35.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-13T10:28:36.309 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:36 smithi060 ceph-mon[27375]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T10:28:36.310 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:36 smithi060 ceph-mon[27375]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T10:28:36.310 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:36 smithi060 ceph-mon[27375]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:28:36.310 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:36.310 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:28:36 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0[41584]: 2024-09-13T10:28:36.169+0000 7f5dfaf9c640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T10:28:36.310 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:28:36 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0[41584]: 2024-09-13T10:28:36.171+0000 7f5df6dc3640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:28:36.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:36 smithi173 ceph-mon[30864]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T10:28:36.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:36 smithi173 ceph-mon[30864]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T10:28:36.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:36 smithi173 ceph-mon[30864]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:28:36.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:36 smithi138 ceph-mon[31033]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T10:28:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:36 smithi138 ceph-mon[31033]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T10:28:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:36 smithi138 ceph-mon[31033]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:28:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:36.981 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:28:37.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:37 smithi060 ceph-mon[27375]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:37.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:37 smithi060 ceph-mon[27375]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:28:37.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:37 smithi060 ceph-mon[27375]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T10:28:37.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:37.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:37 smithi173 ceph-mon[30864]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:37 smithi173 ceph-mon[30864]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:28:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:37 smithi173 ceph-mon[30864]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T10:28:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:37 smithi138 ceph-mon[31033]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T10:28:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:37 smithi138 ceph-mon[31033]: from='osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:28:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:37 smithi138 ceph-mon[31033]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T10:28:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:38.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:38 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:28:38.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:38 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:28:38.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:38 smithi060 ceph-mon[27375]: osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006] boot 2024-09-13T10:28:38.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:38 smithi060 ceph-mon[27375]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T10:28:38.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:38 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:38 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:28:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:38 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:28:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:38 smithi173 ceph-mon[30864]: osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006] boot 2024-09-13T10:28:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:38 smithi173 ceph-mon[30864]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T10:28:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:38 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:38 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:28:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:38 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:28:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:38 smithi138 ceph-mon[31033]: osd.0 [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006] boot 2024-09-13T10:28:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:38 smithi138 ceph-mon[31033]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T10:28:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:38 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-13T10:28:39.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:39.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:39.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T10:28:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:28:39.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:39.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:39.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:28:39.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:40.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:28:40.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2024-09-13T10:28:40.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:40 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi060 2024-09-13T10:28:40.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:40 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi060 to 17092M 2024-09-13T10:28:40.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:40 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi060 2024-09-13T10:28:40.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:40 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi060 to 17092M 2024-09-13T10:28:40.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:40 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi060 2024-09-13T10:28:40.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:40 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi060 to 17092M 2024-09-13T10:28:40.646 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:28:41.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:41 smithi060 ceph-mon[27375]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:41.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:41 smithi173 ceph-mon[30864]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:41 smithi138 ceph-mon[31033]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:42 smithi060 ceph-mon[27375]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:28:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:28:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:42 smithi173 ceph-mon[30864]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:28:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:28:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:42 smithi138 ceph-mon[31033]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:28:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:28:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:28:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:43.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:43 smithi060 ceph-mon[27375]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:43.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:43 smithi173 ceph-mon[30864]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:43.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:43 smithi138 ceph-mon[31033]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:44.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:44 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2641223806' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a"} : dispatch 2024-09-13T10:28:44.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:44 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2641223806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a"}]': finished 2024-09-13T10:28:44.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:44 smithi173 ceph-mon[30864]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T10:28:44.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:44 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:28:44.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:44 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2641223806' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a"} : dispatch 2024-09-13T10:28:44.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:44 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2641223806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a"}]': finished 2024-09-13T10:28:44.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:44 smithi060 ceph-mon[27375]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T10:28:44.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:44 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:28:44.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:44 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2641223806' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a"} : dispatch 2024-09-13T10:28:44.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:44 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2641223806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a"}]': finished 2024-09-13T10:28:44.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:44 smithi138 ceph-mon[31033]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T10:28:44.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:44 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:28:45.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:45 smithi173 ceph-mon[30864]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:45.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:45 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3515656198' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:28:45.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:45 smithi138 ceph-mon[31033]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:45.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:45 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3515656198' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:28:45.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:45 smithi060 ceph-mon[27375]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:45.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:45 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3515656198' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:28:47.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:47 smithi173 ceph-mon[30864]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:47.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:47 smithi138 ceph-mon[31033]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:47.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:47 smithi060 ceph-mon[27375]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:49.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:49 smithi060 ceph-mon[27375]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:49 smithi173 ceph-mon[30864]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:49.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:49 smithi138 ceph-mon[31033]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:51.516 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:51 smithi060 ceph-mon[27375]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:51 smithi173 ceph-mon[30864]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:51.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:51 smithi138 ceph-mon[31033]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:52.293 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-13T10:28:52.293 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:52.294 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:52 smithi060 ceph-mon[27375]: Deploying daemon osd.1 on smithi060 2024-09-13T10:28:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-13T10:28:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:52 smithi173 ceph-mon[30864]: Deploying daemon osd.1 on smithi060 2024-09-13T10:28:52.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-13T10:28:52.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:28:52.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:52 smithi138 ceph-mon[31033]: Deploying daemon osd.1 on smithi060 2024-09-13T10:28:53.463 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:53 smithi060 ceph-mon[27375]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:53.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:53 smithi173 ceph-mon[30864]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:53.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:53 smithi138 ceph-mon[31033]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:55.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:55 smithi173 ceph-mon[30864]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:55.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:55 smithi138 ceph-mon[31033]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:55.601 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:55 smithi060 ceph-mon[27375]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:57.526 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:57 smithi060 ceph-mon[27375]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:57.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:57 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:57.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:57 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:57.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:57 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:57.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:57 smithi173 ceph-mon[30864]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:57.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:57 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:57.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:57 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:57.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:57 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:57 smithi138 ceph-mon[31033]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:57 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:28:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:57 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:57 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:28:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:28:59 smithi173 ceph-mon[30864]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:28:59 smithi138 ceph-mon[31033]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:28:59.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:28:59 smithi060 ceph-mon[27375]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:00.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:00.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:00.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:01.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:01 smithi060 ceph-mon[27375]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:01 smithi173 ceph-mon[30864]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:01.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:01 smithi138 ceph-mon[31033]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:02.238 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 1 on host 'smithi060' 2024-09-13T10:29:02.551 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:02 smithi060 ceph-mon[27375]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-13T10:29:02.551 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:02 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:02.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:02 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:02.552 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:02 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:02.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:02 smithi173 ceph-mon[30864]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-13T10:29:02.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:02 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:02.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:02 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:02.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:02 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:02.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:02 smithi138 ceph-mon[31033]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-13T10:29:02.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:02 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:02.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:02 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:02.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:02 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:02.890 DEBUG:teuthology.orchestra.run.smithi060:osd.1> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.1.service 2024-09-13T10:29:02.892 INFO:tasks.cephadm:Deploying osd.2 on smithi060 with /dev/vg_nvme/lv_2... 2024-09-13T10:29:02.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-13T10:29:03.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-mon[27375]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-mon[27375]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T10:29:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-mon[27375]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T10:29:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-mon[27375]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:03 smithi173 ceph-mon[30864]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:03 smithi173 ceph-mon[30864]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T10:29:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:03 smithi173 ceph-mon[30864]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T10:29:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:03 smithi173 ceph-mon[30864]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:03 smithi138 ceph-mon[31033]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:03 smithi138 ceph-mon[31033]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T10:29:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:03 smithi138 ceph-mon[31033]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T10:29:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:03 smithi138 ceph-mon[31033]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.067 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1[47139]: 2024-09-13T10:29:03.656+0000 7f85483bd640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T10:29:04.067 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:03 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1[47139]: 2024-09-13T10:29:03.659+0000 7f853f9c8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:29:04.559 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:29:04.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:04 smithi060 ceph-mon[27375]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:04.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:04 smithi060 ceph-mon[27375]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T10:29:04.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:04 smithi060 ceph-mon[27375]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' 2024-09-13T10:29:04.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:04 smithi173 ceph-mon[30864]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:04 smithi173 ceph-mon[30864]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T10:29:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:04 smithi173 ceph-mon[30864]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' 2024-09-13T10:29:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:04 smithi138 ceph-mon[31033]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:04 smithi138 ceph-mon[31033]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T10:29:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:04 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:04 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:04 smithi138 ceph-mon[31033]: from='osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161]' entity='osd.1' 2024-09-13T10:29:04.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:04 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:05.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:05 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:29:05.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:05 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:29:05.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:05 smithi060 ceph-mon[27375]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:05.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:05 smithi060 ceph-mon[27375]: osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161] boot 2024-09-13T10:29:05.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:05 smithi060 ceph-mon[27375]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T10:29:05.487 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:05 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:05 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:29:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:05 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:29:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:05 smithi173 ceph-mon[30864]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:05 smithi173 ceph-mon[30864]: osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161] boot 2024-09-13T10:29:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:05 smithi173 ceph-mon[30864]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T10:29:05.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:05 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:05 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:29:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:05 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:29:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:05 smithi138 ceph-mon[31033]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T10:29:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:05 smithi138 ceph-mon[31033]: osd.1 [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161] boot 2024-09-13T10:29:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:05 smithi138 ceph-mon[31033]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T10:29:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:05 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-13T10:29:06.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T10:29:06.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:06.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:06.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:06.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:06.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:06.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:06.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:06.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:06.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-13T10:29:07.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T10:29:07.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:07.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:07.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:07.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:07.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-13T10:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:29:07.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2024-09-13T10:29:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:29:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:07 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi060 2024-09-13T10:29:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:07 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi060 to 8546M 2024-09-13T10:29:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:07 smithi173 ceph-mon[30864]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:07 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T10:29:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:07 smithi173 ceph-mon[30864]: osdmap e15: 2 total, 2 up, 2 in 2024-09-13T10:29:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:07 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-13T10:29:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:07 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi060 2024-09-13T10:29:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:07 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi060 to 8546M 2024-09-13T10:29:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:07 smithi138 ceph-mon[31033]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:07 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T10:29:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:07 smithi138 ceph-mon[31033]: osdmap e15: 2 total, 2 up, 2 in 2024-09-13T10:29:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:07 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-13T10:29:08.116 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:07 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi060 2024-09-13T10:29:08.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:07 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi060 to 8546M 2024-09-13T10:29:08.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:07 smithi060 ceph-mon[27375]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:08.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:07 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T10:29:08.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:07 smithi060 ceph-mon[27375]: osdmap e15: 2 total, 2 up, 2 in 2024-09-13T10:29:08.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:07 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-13T10:29:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50783]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T10:29:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50783]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50783]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50783]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T10:29:08.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50786]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50786]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.374 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50771]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T10:29:08.374 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50771]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.374 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50771]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.374 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50771]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.374 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50774]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50774]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50774]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50774]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50777]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50777]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50777]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50777]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50780]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.375 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:29:08 smithi060 sudo[50780]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T10:29:08.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32750]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32750]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32753]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T10:29:08.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32753]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 sudo[32753]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33009]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T10:29:08.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33009]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33009]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33012]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T10:29:08.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33012]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T10:29:08.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33012]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T10:29:08.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 sudo[33012]: pam_unix(sudo:session): session closed for user root 2024-09-13T10:29:08.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T10:29:08.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: osdmap e16: 2 total, 2 up, 2 in 2024-09-13T10:29:08.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:08.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:29:09.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:09.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T10:29:09.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: osdmap e16: 2 total, 2 up, 2 in 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:29:09.279 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:29:09.280 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:09.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: osdmap e16: 2 total, 2 up, 2 in 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: pgmap v64: 1 pgs: 1 unknown; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:29:09.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:29:09.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:29:09.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:29:09.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:10.107 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: osdmap e17: 2 total, 2 up, 2 in 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: mgrmap e16: y(active, since 116s), standbys: x 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:10.108 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:09 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: osdmap e17: 2 total, 2 up, 2 in 2024-09-13T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: mgrmap e16: y(active, since 116s), standbys: x 2024-09-13T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:10.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:10.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:10.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:09 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:10.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:29:10.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:29:10.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:29:10.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: osdmap e17: 2 total, 2 up, 2 in 2024-09-13T10:29:10.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: mgrmap e16: y(active, since 116s), standbys: x 2024-09-13T10:29:10.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:10.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:10.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:10.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:09 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:11.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:10 smithi060 ceph-mon[27375]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:11.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:10 smithi173 ceph-mon[30864]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:11.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:10 smithi138 ceph-mon[31033]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:11 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2532830646' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87"} : dispatch 2024-09-13T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:11 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2532830646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87"}]': finished 2024-09-13T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:11 smithi173 ceph-mon[30864]: osdmap e18: 3 total, 2 up, 3 in 2024-09-13T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:11 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1233185016' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:29:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:11 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2532830646' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87"} : dispatch 2024-09-13T10:29:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:11 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2532830646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87"}]': finished 2024-09-13T10:29:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:11 smithi138 ceph-mon[31033]: osdmap e18: 3 total, 2 up, 3 in 2024-09-13T10:29:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:11 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1233185016' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:29:12.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:11 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2532830646' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87"} : dispatch 2024-09-13T10:29:12.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:11 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2532830646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87"}]': finished 2024-09-13T10:29:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:11 smithi060 ceph-mon[27375]: osdmap e18: 3 total, 2 up, 3 in 2024-09-13T10:29:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:12.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:11 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1233185016' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:29:13.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:13 smithi060 ceph-mon[27375]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:13 smithi173 ceph-mon[30864]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:13.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:13 smithi138 ceph-mon[31033]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:15.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:15 smithi060 ceph-mon[27375]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:15.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:15 smithi173 ceph-mon[30864]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:15.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:15 smithi138 ceph-mon[31033]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:17.332 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:17 smithi060 ceph-mon[27375]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:17.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:17 smithi173 ceph-mon[30864]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:17.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:17 smithi138 ceph-mon[31033]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:19.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:19 smithi060 ceph-mon[27375]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:19.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:19 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-13T10:29:19.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:19 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:19.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:19 smithi060 ceph-mon[27375]: Deploying daemon osd.2 on smithi060 2024-09-13T10:29:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:19 smithi173 ceph-mon[30864]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:19 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-13T10:29:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:19 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:19 smithi173 ceph-mon[30864]: Deploying daemon osd.2 on smithi060 2024-09-13T10:29:19.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:19 smithi138 ceph-mon[31033]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:19.618 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:19 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-13T10:29:19.618 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:19 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:19.618 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:19 smithi138 ceph-mon[31033]: Deploying daemon osd.2 on smithi060 2024-09-13T10:29:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:21 smithi060 ceph-mon[27375]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:21.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:21 smithi173 ceph-mon[30864]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:21.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:21 smithi138 ceph-mon[31033]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:23 smithi060 ceph-mon[27375]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:23.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:23 smithi173 ceph-mon[30864]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:23.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:23 smithi138 ceph-mon[31033]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:24.450 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:24.450 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:24 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:24.450 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:24 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:24 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:24 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:24 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:24 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:25.234 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:25 smithi060 ceph-mon[27375]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:25.428 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:25 smithi173 ceph-mon[30864]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:25.501 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:25 smithi138 ceph-mon[31033]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:27 smithi173 ceph-mon[30864]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:27 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:27 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:27.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:27 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:27 smithi138 ceph-mon[31033]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:27 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:27 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:27 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:27 smithi060 ceph-mon[27375]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:27 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:27 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:27.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:27 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:28.962 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 2 on host 'smithi060' 2024-09-13T10:29:29.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:29 smithi060 ceph-mon[27375]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:29.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:29 smithi060 ceph-mon[27375]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T10:29:29.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:29.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:29 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:29.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:29 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:29.622 DEBUG:teuthology.orchestra.run.smithi060:osd.2> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.2.service 2024-09-13T10:29:29.624 INFO:tasks.cephadm:Deploying osd.3 on smithi060 with /dev/vg_nvme/lv_1... 2024-09-13T10:29:29.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-13T10:29:29.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:29 smithi173 ceph-mon[30864]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:29.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:29 smithi173 ceph-mon[30864]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T10:29:29.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:29.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:29 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:29.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:29 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:29 smithi138 ceph-mon[31033]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:29 smithi138 ceph-mon[31033]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-13T10:29:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:29 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:29 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:30.712 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:29:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:30 smithi173 ceph-mon[30864]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T10:29:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:30 smithi173 ceph-mon[30864]: osdmap e19: 3 total, 2 up, 3 in 2024-09-13T10:29:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:30 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:30 smithi173 ceph-mon[30864]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:30.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:30 smithi060 ceph-mon[27375]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T10:29:30.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:30 smithi060 ceph-mon[27375]: osdmap e19: 3 total, 2 up, 3 in 2024-09-13T10:29:30.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:30 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:30.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:30 smithi060 ceph-mon[27375]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:30.836 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:29:30 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2[52696]: 2024-09-13T10:29:30.469+0000 7f079ef46640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T10:29:30.836 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:29:30 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2[52696]: 2024-09-13T10:29:30.472+0000 7f079ad6d640 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:29:30.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:30 smithi138 ceph-mon[31033]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T10:29:30.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:30 smithi138 ceph-mon[31033]: osdmap e19: 3 total, 2 up, 3 in 2024-09-13T10:29:30.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:30 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:30.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:30 smithi138 ceph-mon[31033]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:31.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:31 smithi060 ceph-mon[27375]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:31.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:31 smithi060 ceph-mon[27375]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:31.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:31 smithi060 ceph-mon[27375]: osdmap e20: 3 total, 2 up, 3 in 2024-09-13T10:29:31.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:31.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:31.780 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:31 smithi173 ceph-mon[30864]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:31.780 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:31 smithi173 ceph-mon[30864]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:31.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:31 smithi173 ceph-mon[30864]: osdmap e20: 3 total, 2 up, 3 in 2024-09-13T10:29:31.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:31.781 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:31 smithi138 ceph-mon[31033]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 179 GiB / 179 GiB avail 2024-09-13T10:29:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:31 smithi138 ceph-mon[31033]: from='osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:31 smithi138 ceph-mon[31033]: osdmap e20: 3 total, 2 up, 3 in 2024-09-13T10:29:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:32.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:32 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:29:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:32 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:29:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:32 smithi060 ceph-mon[27375]: osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811] boot 2024-09-13T10:29:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:32 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:32 smithi060 ceph-mon[27375]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T10:29:32.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:32 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:29:32.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:32 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:29:32.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:32 smithi173 ceph-mon[30864]: osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811] boot 2024-09-13T10:29:32.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:32 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:32.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:32 smithi173 ceph-mon[30864]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T10:29:32.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:32 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:29:32.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:32 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:29:32.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:32 smithi138 ceph-mon[31033]: osd.2 [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811] boot 2024-09-13T10:29:32.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:32 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-13T10:29:32.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:32 smithi138 ceph-mon[31033]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T10:29:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:33 smithi173 ceph-mon[30864]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:33 smithi173 ceph-mon[30864]: osdmap e22: 3 total, 3 up, 3 in 2024-09-13T10:29:33.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:33 smithi060 ceph-mon[27375]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:33.828 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:33 smithi060 ceph-mon[27375]: osdmap e22: 3 total, 3 up, 3 in 2024-09-13T10:29:33.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:33 smithi138 ceph-mon[31033]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:33.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:33 smithi138 ceph-mon[31033]: osdmap e22: 3 total, 3 up, 3 in 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi060 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.807 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi060 to 5697M 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:34.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:34 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi060 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi060 to 5697M 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:34.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:34.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:34 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:34.908 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:29:34.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2024-09-13T10:29:35.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi060 2024-09-13T10:29:35.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:35.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi060 to 5697M 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:34 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:35.196 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:29:35.586 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:35 smithi060 ceph-mon[27375]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:35.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:35 smithi138 ceph-mon[31033]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:36.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:35 smithi173 ceph-mon[30864]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:36.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:29:36.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:29:36.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:36.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:29:36.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:29:36.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:37.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:29:37.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:29:37.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:37.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:37 smithi060 ceph-mon[27375]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:29:37.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:37 smithi060 ceph-mon[27375]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:37.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:37 smithi138 ceph-mon[31033]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:29:37.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:37 smithi138 ceph-mon[31033]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:38.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:37 smithi173 ceph-mon[30864]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:29:38.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:37 smithi173 ceph-mon[30864]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:38.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:38 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2581514434' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0"} : dispatch 2024-09-13T10:29:38.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:38 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2581514434' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0"}]': finished 2024-09-13T10:29:38.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:38 smithi060 ceph-mon[27375]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T10:29:38.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:38 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:38 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2581514434' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0"} : dispatch 2024-09-13T10:29:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:38 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2581514434' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0"}]': finished 2024-09-13T10:29:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:38 smithi138 ceph-mon[31033]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T10:29:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:38 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:38 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2581514434' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0"} : dispatch 2024-09-13T10:29:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:38 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2581514434' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0"}]': finished 2024-09-13T10:29:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:38 smithi173 ceph-mon[30864]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T10:29:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:38 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:39.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:39 smithi138 ceph-mon[31033]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:39.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:39 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/780660815' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:29:39.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:39 smithi060 ceph-mon[27375]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:39.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:39 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/780660815' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:29:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:39 smithi173 ceph-mon[30864]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:39 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/780660815' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:29:41.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:41 smithi138 ceph-mon[31033]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:41.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:41 smithi060 ceph-mon[27375]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:41 smithi173 ceph-mon[30864]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:42.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:42 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:29:42.699 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:42 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:29:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:42 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:29:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:42 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:29:43.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:42 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:29:43.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:42 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:29:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:43 smithi173 ceph-mon[30864]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:43 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:43 smithi173 ceph-mon[30864]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T10:29:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:43 smithi138 ceph-mon[31033]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:43 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:43.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:43 smithi138 ceph-mon[31033]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T10:29:43.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:43 smithi060 ceph-mon[27375]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:43.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:43 smithi060 ceph-mon[27375]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T10:29:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:45 smithi138 ceph-mon[31033]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-13T10:29:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:45.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:45 smithi060 ceph-mon[27375]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:45.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-13T10:29:45.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:45 smithi173 ceph-mon[30864]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-13T10:29:46.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:46.835 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:46 smithi060 ceph-mon[27375]: Deploying daemon osd.3 on smithi060 2024-09-13T10:29:46.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:46 smithi138 ceph-mon[31033]: Deploying daemon osd.3 on smithi060 2024-09-13T10:29:47.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:46 smithi173 ceph-mon[30864]: Deploying daemon osd.3 on smithi060 2024-09-13T10:29:47.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:47 smithi060 ceph-mon[27375]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:47 smithi138 ceph-mon[31033]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:48.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:47 smithi173 ceph-mon[30864]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:49.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:49 smithi138 ceph-mon[31033]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:49.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:49 smithi060 ceph-mon[27375]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:49 smithi173 ceph-mon[30864]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:51.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:51 smithi138 ceph-mon[31033]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:51.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:51 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:51.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:51 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:51.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:51 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:51 smithi060 ceph-mon[27375]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:51 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:51 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:51 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:51 smithi173 ceph-mon[30864]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:51 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:51 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:51 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:53.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:53 smithi060 ceph-mon[27375]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:53.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:53 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:53 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:53 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:53 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:53 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:53 smithi173 ceph-mon[30864]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:53 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:53 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:53 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:53 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:53 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:53 smithi138 ceph-mon[31033]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:53 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:53 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:53 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:29:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:53 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:29:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:53 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:55.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:55 smithi060 ceph-mon[27375]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:55.902 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:55 smithi138 ceph-mon[31033]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:56.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:55 smithi173 ceph-mon[30864]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:56.425 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 3 on host 'smithi060' 2024-09-13T10:29:56.723 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:56 smithi060 ceph-mon[27375]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-13T10:29:56.723 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:56 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:56.723 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:56 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:56.723 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:56 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:56.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:56 smithi138 ceph-mon[31033]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-13T10:29:56.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:56 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:56.920 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:56 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:56.920 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:56 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:56 smithi173 ceph-mon[30864]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-13T10:29:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:56 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:29:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:56 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:56 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:29:57.042 DEBUG:teuthology.orchestra.run.smithi060:osd.3> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.3.service 2024-09-13T10:29:57.045 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_4... 2024-09-13T10:29:57.045 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-13T10:29:57.299 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:29:57.686 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:57 smithi138 ceph-mon[31033]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:57.686 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:57 smithi138 ceph-mon[31033]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T10:29:57.686 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:57 smithi138 ceph-mon[31033]: osdmap e25: 4 total, 3 up, 4 in 2024-09-13T10:29:57.686 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:57 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:57.686 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:57 smithi138 ceph-mon[31033]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:57.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-mon[27375]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:57.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-mon[27375]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T10:29:57.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-mon[27375]: osdmap e25: 4 total, 3 up, 4 in 2024-09-13T10:29:57.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:57.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-mon[27375]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:57.873 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:29:57.614+0000 7f099d5f8640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T10:29:57.873 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:29:57 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:29:57.619+0000 7f099941f640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:29:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:57 smithi173 ceph-mon[30864]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:57 smithi173 ceph-mon[30864]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T10:29:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:57 smithi173 ceph-mon[30864]: osdmap e25: 4 total, 3 up, 4 in 2024-09-13T10:29:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:57 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:57 smithi173 ceph-mon[30864]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]} : dispatch 2024-09-13T10:29:58.721 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:58 smithi138 ceph-mon[31033]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:58.721 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:58 smithi138 ceph-mon[31033]: osdmap e26: 4 total, 3 up, 4 in 2024-09-13T10:29:58.721 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:58.725 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:58 smithi060 ceph-mon[27375]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:58.725 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:58 smithi060 ceph-mon[27375]: osdmap e26: 4 total, 3 up, 4 in 2024-09-13T10:29:58.725 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:58 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:59.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:58 smithi173 ceph-mon[30864]: from='osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-13T10:29:59.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:58 smithi173 ceph-mon[30864]: osdmap e26: 4 total, 3 up, 4 in 2024-09-13T10:29:59.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:58 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:29:59.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-09-13T10:29:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:59 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:29:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:59 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:29:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:59 smithi060 ceph-mon[27375]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:59 smithi060 ceph-mon[27375]: osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168] boot 2024-09-13T10:29:59.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:29:59 smithi060 ceph-mon[27375]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T10:29:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:59 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:29:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:59 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:29:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:59 smithi138 ceph-mon[31033]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:29:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:29:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:59 smithi138 ceph-mon[31033]: osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168] boot 2024-09-13T10:29:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:29:59 smithi138 ceph-mon[31033]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T10:29:59.899 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:30:00.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:59 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:30:00.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:59 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:30:00.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:59 smithi173 ceph-mon[30864]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-13T10:30:00.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-13T10:30:00.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:59 smithi173 ceph-mon[30864]: osd.3 [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168] boot 2024-09-13T10:30:00.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:29:59 smithi173 ceph-mon[30864]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T10:30:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T10:30:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: overall HEALTH_OK 2024-09-13T10:30:00.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:00.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:00.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:01.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T10:30:01.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:01.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: overall HEALTH_OK 2024-09-13T10:30:01.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:01.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:01.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:01.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: overall HEALTH_OK 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:01.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:01 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi060 2024-09-13T10:30:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:01 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi060 to 4273M 2024-09-13T10:30:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:01 smithi138 ceph-mon[31033]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:01.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:01 smithi138 ceph-mon[31033]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:02.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:01 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi060 2024-09-13T10:30:02.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:01 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi060 to 4273M 2024-09-13T10:30:02.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:01 smithi173 ceph-mon[30864]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:02.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:01 smithi173 ceph-mon[30864]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:02.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:01 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi060 2024-09-13T10:30:02.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:01 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi060 to 4273M 2024-09-13T10:30:02.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:01 smithi060 ceph-mon[27375]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:02.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:01 smithi060 ceph-mon[27375]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/3742513805' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"} : dispatch 2024-09-13T10:30:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"} : dispatch 2024-09-13T10:30:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"}]': finished 2024-09-13T10:30:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T10:30:03.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:03 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/2585347803' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/3742513805' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"} : dispatch 2024-09-13T10:30:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"} : dispatch 2024-09-13T10:30:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"}]': finished 2024-09-13T10:30:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T10:30:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:03 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/2585347803' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:04.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:04.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/3742513805' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"} : dispatch 2024-09-13T10:30:04.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"} : dispatch 2024-09-13T10:30:04.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8"}]': finished 2024-09-13T10:30:04.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:04.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: osdmap e29: 5 total, 4 up, 5 in 2024-09-13T10:30:04.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:03 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/2585347803' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:06.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:05 smithi173 ceph-mon[30864]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:06.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:05 smithi138 ceph-mon[31033]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:06.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:05 smithi060 ceph-mon[27375]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:07 smithi138 ceph-mon[31033]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:07 smithi173 ceph-mon[30864]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:08.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:07 smithi060 ceph-mon[27375]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:09 smithi138 ceph-mon[31033]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:09 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:30:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:09 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:30:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:09 smithi138 ceph-mon[31033]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T10:30:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:09 smithi173 ceph-mon[30864]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:09 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:30:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:09 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:30:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:09 smithi173 ceph-mon[30864]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T10:30:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:10.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:09 smithi060 ceph-mon[27375]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:10.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:09 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:30:10.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:09 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:30:10.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:09 smithi060 ceph-mon[27375]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T10:30:10.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:11.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:11 smithi138 ceph-mon[31033]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:11.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-13T10:30:11.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:11.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:11 smithi138 ceph-mon[31033]: Deploying daemon osd.4 on smithi138 2024-09-13T10:30:11.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:10 smithi173 ceph-mon[30864]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:11.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-13T10:30:11.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:11.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:10 smithi173 ceph-mon[30864]: Deploying daemon osd.4 on smithi138 2024-09-13T10:30:11.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:10 smithi060 ceph-mon[27375]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:11.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-13T10:30:11.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:11.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:10 smithi060 ceph-mon[27375]: Deploying daemon osd.4 on smithi138 2024-09-13T10:30:13.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:13 smithi138 ceph-mon[31033]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:13.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:13 smithi173 ceph-mon[30864]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:13.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:13 smithi060 ceph-mon[27375]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:15.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:15 smithi173 ceph-mon[30864]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:15.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:15 smithi138 ceph-mon[31033]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:15.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:15 smithi060 ceph-mon[27375]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:16 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:16.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:16 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:16.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:16.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:16 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:16.258 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:16 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:16.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:16.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:16 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:16.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:16 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:17.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:17 smithi138 ceph-mon[31033]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:17.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:17 smithi060 ceph-mon[27375]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:17.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:17 smithi173 ceph-mon[30864]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:18 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:18 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:18 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:18 smithi173 ceph-mon[30864]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:18 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:18 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:18 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:18 smithi138 ceph-mon[31033]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:18 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:18 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:18 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:18 smithi060 ceph-mon[27375]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:20.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:19 smithi173 ceph-mon[30864]: from='osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T10:30:20.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:19 smithi173 ceph-mon[30864]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T10:30:20.315 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:19 smithi138 ceph-mon[31033]: from='osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T10:30:20.315 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:19 smithi138 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T10:30:20.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:19 smithi060 ceph-mon[27375]: from='osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T10:30:20.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:19 smithi060 ceph-mon[27375]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-13T10:30:20.618 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-09-13T10:30:21.219 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:21.219 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T10:30:21.219 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T10:30:21.219 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:21.219 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:21.219 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:21.220 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:21.220 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:21.220 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:20 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:21.249 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.4.service 2024-09-13T10:30:21.251 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_3... 2024-09-13T10:30:21.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-13T10:30:21.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:21.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T10:30:21.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T10:30:21.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:21.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:21.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:21.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:21.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:21.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:20 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:21.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:20 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:21.547 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:30:21.411+0000 7fe72c161640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T10:30:21.547 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:30:21.414+0000 7fe72376c640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:30:22.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:21 smithi173 ceph-mon[30864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:30:22.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:21 smithi173 ceph-mon[30864]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T10:30:22.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:21 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:22.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:21 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:22.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:21 smithi173 ceph-mon[30864]: from='osd.4 ' entity='osd.4' 2024-09-13T10:30:22.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:30:22.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-mon[31033]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T10:30:22.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:22.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:22.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:21 smithi138 ceph-mon[31033]: from='osd.4 ' entity='osd.4' 2024-09-13T10:30:22.368 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:30:22.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:21 smithi060 ceph-mon[27375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:30:22.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:21 smithi060 ceph-mon[27375]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T10:30:22.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:21 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:22.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:21 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:22.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:21 smithi060 ceph-mon[27375]: from='osd.4 ' entity='osd.4' 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671] boot 2024-09-13T10:30:23.240 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:22 smithi138 ceph-mon[31033]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T10:30:23.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:30:23.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:30:23.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:23.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:23.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:23.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671] boot 2024-09-13T10:30:23.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:22 smithi173 ceph-mon[30864]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T10:30:23.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:30:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:30:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-09-13T10:30:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-13T10:30:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: osd.4 [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671] boot 2024-09-13T10:30:23.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:22 smithi060 ceph-mon[27375]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T10:30:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:24 smithi138 ceph-mon[31033]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T10:30:24.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:24 smithi173 ceph-mon[30864]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T10:30:24.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:24 smithi060 ceph-mon[27375]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T10:30:25.768 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:25.768 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi138 2024-09-13T10:30:25.768 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:25 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:26.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi138 2024-09-13T10:30:26.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:26.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:26.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-13T10:30:26.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:26.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:26.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:25 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:26.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi138 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:26.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:25 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:30:26.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-09-13T10:30:26.431 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:30:27.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:27 smithi138 ceph-mon[31033]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:27.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:27.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:27.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:28.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:27 smithi173 ceph-mon[30864]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:28.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:28.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:28.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:28.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:27 smithi060 ceph-mon[27375]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:28.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:28.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:28.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:28.770 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:28 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:30:28.770 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:28 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:30:28.770 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:28 smithi138 ceph-mon[31033]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:29.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:28 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:30:29.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:28 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:30:29.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:28 smithi173 ceph-mon[30864]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:29.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:28 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:30:29.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:28 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:30:29.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:28 smithi060 ceph-mon[27375]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:30.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:29 smithi173 ceph-mon[30864]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:30.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:29 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/1654667658' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"} : dispatch 2024-09-13T10:30:30.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:29 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"} : dispatch 2024-09-13T10:30:30.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:29 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"}]': finished 2024-09-13T10:30:30.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:30.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:29 smithi173 ceph-mon[30864]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T10:30:30.061 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:29 smithi138 ceph-mon[31033]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:30.061 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:29 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/1654667658' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"} : dispatch 2024-09-13T10:30:30.061 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:29 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"} : dispatch 2024-09-13T10:30:30.061 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:29 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"}]': finished 2024-09-13T10:30:30.061 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:30.061 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:29 smithi138 ceph-mon[31033]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T10:30:30.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:29 smithi060 ceph-mon[27375]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:30.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:29 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/1654667658' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"} : dispatch 2024-09-13T10:30:30.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:29 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"} : dispatch 2024-09-13T10:30:30.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:29 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9037c1e7-d476-4980-8155-6244430c9011"}]': finished 2024-09-13T10:30:30.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:30.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:29 smithi060 ceph-mon[27375]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T10:30:31.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:31 smithi173 ceph-mon[30864]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:31.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:31 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/1822573819' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:31.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:31 smithi138 ceph-mon[31033]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:31.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:31 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/1822573819' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:31.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:31 smithi060 ceph-mon[27375]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:31.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:31 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/1822573819' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:33.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:33 smithi173 ceph-mon[30864]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:33.305 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:33 smithi138 ceph-mon[31033]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:33.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:33 smithi060 ceph-mon[27375]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:35.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:35 smithi173 ceph-mon[30864]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:35.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:35 smithi138 ceph-mon[31033]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:35.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:35 smithi060 ceph-mon[27375]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:37.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:37 smithi138 ceph-mon[31033]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:37.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-13T10:30:37.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:37.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:37 smithi173 ceph-mon[30864]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:37.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-13T10:30:37.278 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:37.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:37 smithi060 ceph-mon[27375]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:37.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-13T10:30:37.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:38.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:38 smithi138 ceph-mon[31033]: Deploying daemon osd.5 on smithi138 2024-09-13T10:30:38.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:38 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:30:38.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:38 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:30:38.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:38 smithi173 ceph-mon[30864]: Deploying daemon osd.5 on smithi138 2024-09-13T10:30:38.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:38 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:30:38.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:38 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:30:38.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:38 smithi060 ceph-mon[27375]: Deploying daemon osd.5 on smithi138 2024-09-13T10:30:38.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:38 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:30:38.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:38 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:30:39.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:39 smithi173 ceph-mon[30864]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:39.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:39 smithi173 ceph-mon[30864]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T10:30:39.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:39.313 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:39 smithi138 ceph-mon[31033]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:39.313 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:39 smithi138 ceph-mon[31033]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T10:30:39.313 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:39.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:39 smithi060 ceph-mon[27375]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:39.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:39 smithi060 ceph-mon[27375]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T10:30:39.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:41.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:41 smithi138 ceph-mon[31033]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:41.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:41 smithi173 ceph-mon[30864]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:41.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:41 smithi060 ceph-mon[27375]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:42.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:42.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:42 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:42 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:42 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:43 smithi138 ceph-mon[31033]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:43 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:43.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:43 smithi060 ceph-mon[27375]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:43.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:43 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:43.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:43 smithi173 ceph-mon[30864]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:43.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:43 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:45.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:45 smithi060 ceph-mon[27375]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:45 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:45 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:45 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:45 smithi173 ceph-mon[30864]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:45 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:45 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:45 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:46.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:45 smithi138 ceph-mon[31033]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:46.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:45 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:46.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:45 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:46.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:46.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:46.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:45 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:47.580 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-09-13T10:30:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:47 smithi138 ceph-mon[31033]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:47 smithi138 ceph-mon[31033]: from='osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T10:30:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:47 smithi138 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T10:30:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:47 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:47 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:47.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:47 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:48.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:47 smithi173 ceph-mon[30864]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:48.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:47 smithi173 ceph-mon[30864]: from='osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T10:30:48.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:47 smithi173 ceph-mon[30864]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T10:30:48.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:47 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:48.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:47 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:48.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:47 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:48.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:47 smithi060 ceph-mon[27375]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:48.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:47 smithi060 ceph-mon[27375]: from='osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T10:30:48.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:47 smithi060 ceph-mon[27375]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-13T10:30:48.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:47 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:48.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:47 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:48.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:47 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:48.277 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.5.service 2024-09-13T10:30:48.280 INFO:tasks.cephadm:Deploying osd.6 on smithi138 with /dev/vg_nvme/lv_2... 2024-09-13T10:30:48.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-13T10:30:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T10:30:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-mon[31033]: from='osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-mon[31033]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T10:30:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:48.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:30:48.633+0000 7f81d3aee640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T10:30:48.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:30:48 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:30:48.638+0000 7f81cf915640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:30:49.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:48 smithi173 ceph-mon[30864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T10:30:49.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:48 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:49.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:48 smithi173 ceph-mon[30864]: from='osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:49.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:48 smithi173 ceph-mon[30864]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T10:30:49.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:48 smithi173 ceph-mon[30864]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:49.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:48 smithi060 ceph-mon[27375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T10:30:49.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:48 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:49.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:48 smithi060 ceph-mon[27375]: from='osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:49.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:48 smithi060 ceph-mon[27375]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T10:30:49.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:48 smithi060 ceph-mon[27375]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:30:49.364 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:30:49.950 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:49 smithi138 ceph-mon[31033]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:49.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:49 smithi138 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:30:49.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:49 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:49.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:49 smithi138 ceph-mon[31033]: osdmap e38: 6 total, 5 up, 6 in 2024-09-13T10:30:49.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:49 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:49 smithi173 ceph-mon[30864]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:49 smithi173 ceph-mon[30864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:30:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:49 smithi173 ceph-mon[30864]: osdmap e38: 6 total, 5 up, 6 in 2024-09-13T10:30:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:49 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:50.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:49 smithi060 ceph-mon[27375]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:50.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:49 smithi060 ceph-mon[27375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:30:50.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:49 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:50.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:49 smithi060 ceph-mon[27375]: osdmap e38: 6 total, 5 up, 6 in 2024-09-13T10:30:50.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:49 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:50 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:30:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:50 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:30:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:50 smithi138 ceph-mon[31033]: osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914] boot 2024-09-13T10:30:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:50 smithi138 ceph-mon[31033]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T10:30:50.948 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:50 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:51.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:50 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:30:51.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:50 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:30:51.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:50 smithi173 ceph-mon[30864]: osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914] boot 2024-09-13T10:30:51.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:50 smithi173 ceph-mon[30864]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T10:30:51.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:50 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:51.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:50 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:30:51.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:50 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:30:51.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:50 smithi060 ceph-mon[27375]: osd.5 [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914] boot 2024-09-13T10:30:51.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:50 smithi060 ceph-mon[27375]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T10:30:51.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:50 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-13T10:30:51.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:51 smithi138 ceph-mon[31033]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:51.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:51 smithi138 ceph-mon[31033]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T10:30:51.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:51 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:51.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:51 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:51.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:51 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:51.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:51 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:51 smithi173 ceph-mon[30864]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:51 smithi173 ceph-mon[30864]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T10:30:52.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:51 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:52.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:51 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:52.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:51 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:52.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:51 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:52.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:51 smithi060 ceph-mon[27375]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-09-13T10:30:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:51 smithi060 ceph-mon[27375]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T10:30:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:51 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:51 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:30:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:51 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:51 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:52.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi138 2024-09-13T10:30:52.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:30:52.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:30:52.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:30:52.918 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:52.919 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:52.921 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:30:52.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-09-13T10:30:53.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi138 2024-09-13T10:30:53.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:30:53.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:30:53.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:30:53.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:53.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:53.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:53.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi138 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:53.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:30:53.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:30:53.192 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:30:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:53 smithi138 ceph-mon[31033]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:30:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:53 smithi138 ceph-mon[31033]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-13T10:30:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:53 smithi173 ceph-mon[30864]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:30:54.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:53 smithi173 ceph-mon[30864]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-13T10:30:54.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:53 smithi060 ceph-mon[27375]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:30:54.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:53 smithi060 ceph-mon[27375]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-13T10:30:54.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:54 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:54.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:54 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:54.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:54 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:55.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:55.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:55.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:54 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:55.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:30:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:30:55.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:54 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:30:55.834 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:55 smithi138 ceph-mon[31033]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:55.834 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:55 smithi138 ceph-mon[31033]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:30:55.928 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:55 smithi173 ceph-mon[30864]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:55.928 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:55 smithi173 ceph-mon[30864]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:30:56.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:55 smithi060 ceph-mon[27375]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:30:56.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:55 smithi060 ceph-mon[27375]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:30:56.914 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:56 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/2507712418' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"} : dispatch 2024-09-13T10:30:56.914 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:56 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"} : dispatch 2024-09-13T10:30:56.914 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:56 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"}]': finished 2024-09-13T10:30:56.914 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:56 smithi138 ceph-mon[31033]: osdmap e41: 7 total, 6 up, 7 in 2024-09-13T10:30:56.914 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:56 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:30:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:56 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/2507712418' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"} : dispatch 2024-09-13T10:30:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:56 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"} : dispatch 2024-09-13T10:30:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:56 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"}]': finished 2024-09-13T10:30:57.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:56 smithi173 ceph-mon[30864]: osdmap e41: 7 total, 6 up, 7 in 2024-09-13T10:30:57.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:56 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:30:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:56 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/2507712418' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"} : dispatch 2024-09-13T10:30:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:56 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"} : dispatch 2024-09-13T10:30:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:56 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b635470b-4619-4a8a-b016-1811d5049544"}]': finished 2024-09-13T10:30:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:56 smithi060 ceph-mon[27375]: osdmap e41: 7 total, 6 up, 7 in 2024-09-13T10:30:57.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:56 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:30:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:57 smithi173 ceph-mon[30864]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-13T10:30:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:57 smithi173 ceph-mon[30864]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-13T10:30:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:57 smithi173 ceph-mon[30864]: Cluster is now healthy 2024-09-13T10:30:58.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:57 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/33124622' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:58.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:57 smithi138 ceph-mon[31033]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-13T10:30:58.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:57 smithi138 ceph-mon[31033]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-13T10:30:58.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:57 smithi138 ceph-mon[31033]: Cluster is now healthy 2024-09-13T10:30:58.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:57 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/33124622' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:58.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:57 smithi060 ceph-mon[27375]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-13T10:30:58.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:57 smithi060 ceph-mon[27375]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-13T10:30:58.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:57 smithi060 ceph-mon[27375]: Cluster is now healthy 2024-09-13T10:30:58.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:57 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/33124622' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:30:59.968 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:30:59 smithi138 ceph-mon[31033]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:31:00.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:30:59 smithi173 ceph-mon[30864]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:31:00.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:30:59 smithi060 ceph-mon[27375]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:31:02.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:01 smithi173 ceph-mon[30864]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T10:31:02.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:01 smithi138 ceph-mon[31033]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T10:31:02.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:01 smithi060 ceph-mon[27375]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T10:31:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:03 smithi138 ceph-mon[31033]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-13T10:31:03.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:03 smithi060 ceph-mon[27375]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-13T10:31:03.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:03 smithi173 ceph-mon[30864]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:04.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-13T10:31:04.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:04.997 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:04 smithi138 ceph-mon[31033]: Deploying daemon osd.6 on smithi138 2024-09-13T10:31:05.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:04 smithi173 ceph-mon[30864]: Deploying daemon osd.6 on smithi138 2024-09-13T10:31:05.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:04 smithi060 ceph-mon[27375]: Deploying daemon osd.6 on smithi138 2024-09-13T10:31:05.997 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:05 smithi138 ceph-mon[31033]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:06.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:05 smithi173 ceph-mon[30864]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:06.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:05 smithi060 ceph-mon[27375]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:07.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:07 smithi138 ceph-mon[31033]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:07 smithi173 ceph-mon[30864]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:08.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:07 smithi060 ceph-mon[27375]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:09.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:08 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:09.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:08 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:09.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:08 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:10.018 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:09 smithi138 ceph-mon[31033]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:10.018 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:10.018 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:09 smithi173 ceph-mon[30864]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:10.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:10.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:09 smithi060 ceph-mon[27375]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:10.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:10.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:11 smithi173 ceph-mon[30864]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:12.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:12.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:12.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:11 smithi138 ceph-mon[31033]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:12.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:12.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:12.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:11 smithi060 ceph-mon[27375]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:12.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:12.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:12.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:12.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:13 smithi138 ceph-mon[31033]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:13 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:13 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:13 smithi138 ceph-mon[31033]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T10:31:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:14.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:13 smithi173 ceph-mon[30864]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:14.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:13 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:14.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:13 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:14.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:13 smithi173 ceph-mon[30864]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T10:31:14.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:14.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:13 smithi060 ceph-mon[27375]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:14.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:13 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:14.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:13 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:14.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:13 smithi060 ceph-mon[27375]: osdmap e42: 7 total, 6 up, 7 in 2024-09-13T10:31:14.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:14.193 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 6 on host 'smithi138' 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: from='osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:14.791 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:14.819 DEBUG:teuthology.orchestra.run.smithi138:osd.6> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.6.service 2024-09-13T10:31:14.821 INFO:tasks.cephadm:Deploying osd.7 on smithi138 with /dev/vg_nvme/lv_1... 2024-09-13T10:31:14.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-13T10:31:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T10:31:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:15.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: from='osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T10:31:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T10:31:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:15.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:15.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: from='osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:15.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:15.569 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:31:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:15 smithi138 ceph-mon[31033]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:15 smithi138 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T10:31:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:15 smithi138 ceph-mon[31033]: from='osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:15 smithi138 ceph-mon[31033]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T10:31:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:15 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:15 smithi138 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:15 smithi173 ceph-mon[30864]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:15 smithi173 ceph-mon[30864]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T10:31:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:15 smithi173 ceph-mon[30864]: from='osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:15 smithi173 ceph-mon[30864]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T10:31:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:15 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:16.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:15 smithi173 ceph-mon[30864]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:16.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:15 smithi060 ceph-mon[27375]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:16.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:15 smithi060 ceph-mon[27375]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T10:31:16.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:15 smithi060 ceph-mon[27375]: from='osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:16.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:15 smithi060 ceph-mon[27375]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T10:31:16.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:15 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:16.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:15 smithi060 ceph-mon[27375]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:16.318 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:31:16.148+0000 7f400de3d640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T10:31:16.319 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:31:16.155+0000 7f4005c5c640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:31:16.862 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:31:16.862 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-mon[31033]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T10:31:16.862 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:16.862 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:16.862 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:16 smithi138 ceph-mon[31033]: from='osd.6 ' entity='osd.6' 2024-09-13T10:31:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:16 smithi173 ceph-mon[30864]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:31:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:16 smithi173 ceph-mon[30864]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T10:31:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:17.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:16 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:17.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:16 smithi173 ceph-mon[30864]: from='osd.6 ' entity='osd.6' 2024-09-13T10:31:17.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:16 smithi060 ceph-mon[27375]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:31:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:16 smithi060 ceph-mon[27375]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T10:31:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:16 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:17.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:16 smithi060 ceph-mon[27375]: from='osd.6 ' entity='osd.6' 2024-09-13T10:31:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312] boot 2024-09-13T10:31:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T10:31:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:17.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:17.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:17 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312] boot 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:18.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:18.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:17 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:18.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: osd.6 [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312] boot 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:18.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:17 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:18.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:18 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T10:31:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:19.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:18 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:19.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:18 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:19.716 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-13T10:31:19.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-09-13T10:31:19.767 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:19 smithi138 ceph-mon[31033]: pgmap v160: 1 pgs: 1 remapped; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:19.767 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:19 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi138 2024-09-13T10:31:19.768 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:19 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-13T10:31:19.768 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:19 smithi138 ceph-mon[31033]: osdmap e48: 7 total, 7 up, 7 in 2024-09-13T10:31:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:31:20.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:19 smithi173 ceph-mon[30864]: pgmap v160: 1 pgs: 1 remapped; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:20.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:19 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi138 2024-09-13T10:31:20.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:19 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-13T10:31:20.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:19 smithi173 ceph-mon[30864]: osdmap e48: 7 total, 7 up, 7 in 2024-09-13T10:31:20.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:19 smithi060 ceph-mon[27375]: pgmap v160: 1 pgs: 1 remapped; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:20.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:19 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi138 2024-09-13T10:31:20.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:19 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-13T10:31:20.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:19 smithi060 ceph-mon[27375]: osdmap e48: 7 total, 7 up, 7 in 2024-09-13T10:31:21.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:31:21.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:31:21.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:31:21.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:21.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:21.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:21.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:21.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:31:21.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:31:21.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:20 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:31:21.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:20 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:21.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:31:21.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:20 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:22.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:21 smithi173 ceph-mon[30864]: pgmap v162: 1 pgs: 1 remapped; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:22.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:21 smithi173 ceph-mon[30864]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:31:22.056 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:21 smithi138 ceph-mon[31033]: pgmap v162: 1 pgs: 1 remapped; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:22.056 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:21 smithi138 ceph-mon[31033]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:31:22.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:21 smithi060 ceph-mon[27375]: pgmap v162: 1 pgs: 1 remapped; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:22.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:21 smithi060 ceph-mon[27375]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:31:24.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: pgmap v163: 1 pgs: 1 remapped; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:24.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/2522242291' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"} : dispatch 2024-09-13T10:31:24.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"} : dispatch 2024-09-13T10:31:24.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"}]': finished 2024-09-13T10:31:24.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T10:31:24.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:24.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:23 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/2994159239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:31:24.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: pgmap v163: 1 pgs: 1 remapped; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:24.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/2522242291' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"} : dispatch 2024-09-13T10:31:24.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"} : dispatch 2024-09-13T10:31:24.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"}]': finished 2024-09-13T10:31:24.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T10:31:24.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:24.102 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:23 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/2994159239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:31:24.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: pgmap v163: 1 pgs: 1 remapped; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:24.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/2522242291' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"} : dispatch 2024-09-13T10:31:24.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"} : dispatch 2024-09-13T10:31:24.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3"}]': finished 2024-09-13T10:31:24.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T10:31:24.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:24.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:23 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/2994159239' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:31:26.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:25 smithi173 ceph-mon[30864]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T10:31:26.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:25 smithi138 ceph-mon[31033]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T10:31:26.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:25 smithi060 ceph-mon[27375]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-13T10:31:27.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:26 smithi138 ceph-mon[31033]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:31:27.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:26 smithi060 ceph-mon[27375]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:31:27.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:26 smithi173 ceph-mon[30864]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:31:29.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:28 smithi138 ceph-mon[31033]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T10:31:29.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:28 smithi173 ceph-mon[30864]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T10:31:29.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:28 smithi060 ceph-mon[27375]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-13T10:31:31.067 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:31 smithi138 ceph-mon[31033]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:31.067 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-13T10:31:31.067 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:31.068 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:31 smithi138 ceph-mon[31033]: Deploying daemon osd.7 on smithi138 2024-09-13T10:31:31.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:31 smithi173 ceph-mon[30864]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:31.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-13T10:31:31.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:31.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:31 smithi173 ceph-mon[30864]: Deploying daemon osd.7 on smithi138 2024-09-13T10:31:31.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:31 smithi060 ceph-mon[27375]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:31.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-13T10:31:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:31 smithi060 ceph-mon[27375]: Deploying daemon osd.7 on smithi138 2024-09-13T10:31:33.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:33 smithi173 ceph-mon[30864]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:33.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:33 smithi138 ceph-mon[31033]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:33.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:33 smithi060 ceph-mon[27375]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T10:31:35.263 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:35 smithi138 ceph-mon[31033]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:35.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:35 smithi173 ceph-mon[30864]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:35.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:35 smithi060 ceph-mon[27375]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:36.276 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:36.276 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:36.276 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:36.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:36.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:36.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:36.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:36.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:36.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:37.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:37 smithi173 ceph-mon[30864]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:37.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:37 smithi138 ceph-mon[31033]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:37.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:37 smithi060 ceph-mon[27375]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:39.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:39 smithi060 ceph-mon[27375]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:39.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:39.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:39.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:39.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:39.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:39 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:39 smithi173 ceph-mon[30864]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:40.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:40.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:39 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:39 smithi138 ceph-mon[31033]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:40.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:40.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:40.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:39 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:40.734 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:40 smithi138 ceph-mon[31033]: from='osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T10:31:40.735 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:40 smithi138 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T10:31:40.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:40 smithi060 ceph-mon[27375]: from='osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T10:31:40.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:40 smithi060 ceph-mon[27375]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T10:31:41.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:40 smithi173 ceph-mon[30864]: from='osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T10:31:41.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:40 smithi173 ceph-mon[30864]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-13T10:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 7 on host 'smithi138' 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:41.640 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:41.641 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:41.641 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:41 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:41.665 DEBUG:teuthology.orchestra.run.smithi138:osd.7> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.7.service 2024-09-13T10:31:41.707 INFO:tasks.cephadm:Deploying osd.8 on smithi173 with /dev/vg_nvme/lv_4... 2024-09-13T10:31:41.707 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-13T10:31:41.963 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:31:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T10:31:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T10:31:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:42.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:42.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:42.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:42.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:41 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:42.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:41 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:42.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:42 smithi138 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:31:42.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:42 smithi138 ceph-mon[31033]: osdmap e51: 8 total, 7 up, 8 in 2024-09-13T10:31:42.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:42.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:43.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:42 smithi173 ceph-mon[30864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:31:43.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:42 smithi173 ceph-mon[30864]: osdmap e51: 8 total, 7 up, 8 in 2024-09-13T10:31:43.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:43.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:43.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:42 smithi060 ceph-mon[27375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-13T10:31:43.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:42 smithi060 ceph-mon[27375]: osdmap e51: 8 total, 7 up, 8 in 2024-09-13T10:31:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:43.634 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:43.635 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:43.635 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:43.635 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744] boot 2024-09-13T10:31:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744] boot 2024-09-13T10:31:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T10:31:43.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:43.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:43 smithi138 ceph-mon[31033]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T10:31:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T10:31:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:43 smithi173 ceph-mon[30864]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T10:31:44.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-13T10:31:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: osd.7 [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744] boot 2024-09-13T10:31:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T10:31:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-13T10:31:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:43 smithi060 ceph-mon[27375]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T10:31:44.281 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:31:44.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-09-13T10:31:44.565 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:31:45.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:45.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi138 2024-09-13T10:31:45.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T10:31:45.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:31:45.646 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:31:45.646 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:45 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi138 2024-09-13T10:31:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:31:45.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:31:45.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:45 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:45.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi138 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:31:45.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T10:31:45.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:31:45.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:31:45.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:45 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:46.562 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:46 smithi173 ceph-mon[30864]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:31:46.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:46 smithi138 ceph-mon[31033]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:31:46.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:46 smithi060 ceph-mon[27375]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:31:47.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:47 smithi173 ceph-mon[30864]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:47 smithi138 ceph-mon[31033]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:47 smithi060 ceph-mon[27375]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:48 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/3616306311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"} : dispatch 2024-09-13T10:31:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:48 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"} : dispatch 2024-09-13T10:31:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:48 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"}]': finished 2024-09-13T10:31:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:48 smithi173 ceph-mon[30864]: osdmap e55: 9 total, 8 up, 9 in 2024-09-13T10:31:48.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:48 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:48.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:48 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/2002805189' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:31:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:48 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/3616306311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"} : dispatch 2024-09-13T10:31:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:48 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"} : dispatch 2024-09-13T10:31:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:48 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"}]': finished 2024-09-13T10:31:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:48 smithi138 ceph-mon[31033]: osdmap e55: 9 total, 8 up, 9 in 2024-09-13T10:31:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:48 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:48.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:48 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/2002805189' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:31:48.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:48 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/3616306311' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"} : dispatch 2024-09-13T10:31:48.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:48 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"} : dispatch 2024-09-13T10:31:48.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:48 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667"}]': finished 2024-09-13T10:31:48.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:48 smithi060 ceph-mon[27375]: osdmap e55: 9 total, 8 up, 9 in 2024-09-13T10:31:48.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:48 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:48.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:48 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/2002805189' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:31:49.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:49 smithi173 ceph-mon[30864]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:49.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:49 smithi138 ceph-mon[31033]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:49.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:49 smithi060 ceph-mon[27375]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:51.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:51 smithi173 ceph-mon[30864]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:51.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:51 smithi138 ceph-mon[31033]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:51.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:51 smithi060 ceph-mon[27375]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:52.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:52 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:52.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:52 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:52.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:52 smithi173 ceph-mon[30864]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T10:31:52.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:52 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:52.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:52 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:52.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:52 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:52.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:52 smithi138 ceph-mon[31033]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T10:31:52.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:52 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:52.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:52 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:52.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:52 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:52 smithi060 ceph-mon[27375]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T10:31:52.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:52 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:53 smithi173 ceph-mon[30864]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:53 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:31:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:53 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:31:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:53 smithi173 ceph-mon[30864]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T10:31:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:53 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:53 smithi138 ceph-mon[31033]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:53 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:31:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:53 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:31:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:53 smithi138 ceph-mon[31033]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T10:31:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:53 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:53.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:53 smithi060 ceph-mon[27375]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:53 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:31:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:53 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:31:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:53 smithi060 ceph-mon[27375]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T10:31:53.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:53 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:31:55.847 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:55 smithi173 ceph-mon[30864]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:55.848 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:55 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-13T10:31:55.848 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:55 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:55.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:55 smithi138 ceph-mon[31033]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:55.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:55 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-13T10:31:55.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:55 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:55.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:55 smithi060 ceph-mon[27375]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:55.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:55 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-13T10:31:55.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:55 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:31:56.788 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:56 smithi173 ceph-mon[30864]: Deploying daemon osd.8 on smithi173 2024-09-13T10:31:56.848 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:56 smithi138 ceph-mon[31033]: Deploying daemon osd.8 on smithi173 2024-09-13T10:31:56.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:56 smithi060 ceph-mon[27375]: Deploying daemon osd.8 on smithi173 2024-09-13T10:31:57.648 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:57 smithi173 ceph-mon[30864]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:57.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:57 smithi138 ceph-mon[31033]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:57.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:57 smithi060 ceph-mon[27375]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:59.734 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:31:59 smithi173 ceph-mon[30864]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:31:59 smithi138 ceph-mon[31033]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:31:59.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:31:59 smithi060 ceph-mon[27375]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:00 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:32:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:00 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:32:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:00 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:32:00.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:00 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:32:00.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:00 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:32:00.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:00 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:32:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:01 smithi173 ceph-mon[30864]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:01 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:01 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:01 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:01.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:01 smithi138 ceph-mon[31033]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:01.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:01 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:01.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:01 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:01.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:01 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:01.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:01 smithi060 ceph-mon[27375]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:01.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:01 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:01.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:01 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:01.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:01 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:03.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:03.779 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:04 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:04 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:04 smithi138 ceph-mon[31033]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T10:32:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:04 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:04.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:04 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:04.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:04 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:04 smithi060 ceph-mon[27375]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T10:32:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:04 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:04 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:04 smithi173 ceph-mon[30864]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:05 smithi138 ceph-mon[31033]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:05 smithi138 ceph-mon[31033]: from='osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-13T10:32:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:05 smithi138 ceph-mon[31033]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-13T10:32:05.864 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:05 smithi173 ceph-mon[30864]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:05.864 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:05 smithi173 ceph-mon[30864]: from='osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-13T10:32:05.864 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:05 smithi173 ceph-mon[30864]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-13T10:32:05.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:05 smithi060 ceph-mon[27375]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:05.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:05 smithi060 ceph-mon[27375]: from='osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-13T10:32:05.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:05 smithi060 ceph-mon[27375]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-13T10:32:06.196 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 8 on host 'smithi173' 2024-09-13T10:32:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T10:32:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: osdmap e59: 9 total, 8 up, 9 in 2024-09-13T10:32:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:06.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:06.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:06.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:06 smithi173 ceph-mon[30864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:32:06.841 DEBUG:teuthology.orchestra.run.smithi173:osd.8> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.8.service 2024-09-13T10:32:06.843 INFO:tasks.cephadm:Deploying osd.9 on smithi173 with /dev/vg_nvme/lv_3... 2024-09-13T10:32:06.843 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-13T10:32:07.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: osdmap e59: 9 total, 8 up, 9 in 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:07.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:06 smithi138 ceph-mon[31033]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:32:07.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T10:32:07.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: osdmap e59: 9 total, 8 up, 9 in 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:07.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:06 smithi060 ceph-mon[27375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:32:07.174 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:32:07.001+0000 7f9d2e831640 -1 osd.8 0 waiting for initial osdmap 2024-09-13T10:32:07.174 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:32:07.009+0000 7f9d25e3c640 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:32:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: osdmap e60: 9 total, 8 up, 9 in 2024-09-13T10:32:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: from='osd.8 ' entity='osd.8' 2024-09-13T10:32:08.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558] boot 2024-09-13T10:32:08.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T10:32:08.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:07 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.082 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:32:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: osdmap e60: 9 total, 8 up, 9 in 2024-09-13T10:32:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: from='osd.8 ' entity='osd.8' 2024-09-13T10:32:08.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558] boot 2024-09-13T10:32:08.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T10:32:08.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:07 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: osdmap e60: 9 total, 8 up, 9 in 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: from='osd.8 ' entity='osd.8' 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: osd.8 [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558] boot 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T10:32:08.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:07 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-13T10:32:09.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:08 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:09.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:08 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:09.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:08 smithi173 ceph-mon[30864]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T10:32:09.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:08 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:09.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:08 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:09.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:08 smithi138 ceph-mon[31033]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T10:32:09.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:08 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:09.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:08 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:09.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:08 smithi060 ceph-mon[27375]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T10:32:09.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:09 smithi173 ceph-mon[30864]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:09.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:09 smithi173 ceph-mon[30864]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T10:32:10.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:09 smithi138 ceph-mon[31033]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:10.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:09 smithi138 ceph-mon[31033]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T10:32:10.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:09 smithi060 ceph-mon[27375]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:10.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:09 smithi060 ceph-mon[27375]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T10:32:11.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi173 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi173 to 21184M 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:11.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:11.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.693 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:11.693 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi173 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi173 to 21184M 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:11.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.696 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:32:11.724 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-09-13T10:32:11.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi173 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi173 to 21184M 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:11.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:11.967 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:32:13.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:13 smithi060 ceph-mon[27375]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:13.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:13 smithi060 ceph-mon[27375]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:32:13.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:32:13.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:32:13.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:13 smithi173 ceph-mon[30864]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:13 smithi173 ceph-mon[30864]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:32:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:32:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:32:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:13.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:13 smithi138 ceph-mon[31033]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:13.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:13 smithi138 ceph-mon[31033]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:32:13.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:32:13.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:32:13.853 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:15.496 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:15 smithi173 ceph-mon[30864]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-13T10:32:15.497 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:15 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/3709344374' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"} : dispatch 2024-09-13T10:32:15.497 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:15 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"} : dispatch 2024-09-13T10:32:15.497 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:15 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"}]': finished 2024-09-13T10:32:15.497 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:15 smithi173 ceph-mon[30864]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T10:32:15.497 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:15 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:15 smithi138 ceph-mon[31033]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-13T10:32:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:15 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/3709344374' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"} : dispatch 2024-09-13T10:32:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:15 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"} : dispatch 2024-09-13T10:32:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:15 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"}]': finished 2024-09-13T10:32:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:15 smithi138 ceph-mon[31033]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T10:32:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:15 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:15 smithi060 ceph-mon[27375]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-13T10:32:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:15 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/3709344374' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"} : dispatch 2024-09-13T10:32:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:15 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"} : dispatch 2024-09-13T10:32:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:15 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af"}]': finished 2024-09-13T10:32:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:15 smithi060 ceph-mon[27375]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T10:32:15.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:15 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:16.681 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:16 smithi173 ceph-mon[30864]: from='client.24295 172.21.15.173:0/2363304626' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:32:16.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:16 smithi138 ceph-mon[31033]: from='client.24295 172.21.15.173:0/2363304626' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:32:16.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:16 smithi060 ceph-mon[27375]: from='client.24295 172.21.15.173:0/2363304626' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:32:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:17 smithi173 ceph-mon[30864]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T10:32:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:17 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:17 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:17 smithi173 ceph-mon[30864]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T10:32:17.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:17 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:17 smithi138 ceph-mon[31033]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T10:32:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:17 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:17 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:17 smithi138 ceph-mon[31033]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T10:32:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:17 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:17.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:17 smithi060 ceph-mon[27375]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T10:32:17.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:17 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:17 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:17 smithi060 ceph-mon[27375]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T10:32:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:17 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:19 smithi173 ceph-mon[30864]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:32:19.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:19 smithi138 ceph-mon[31033]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:32:19.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:19 smithi060 ceph-mon[27375]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:32:21.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:21 smithi173 ceph-mon[30864]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:32:21.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:21 smithi138 ceph-mon[31033]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:32:21.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:21 smithi060 ceph-mon[27375]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:32:22.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:22 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-13T10:32:22.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:22 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:22.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:22 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-13T10:32:22.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:22 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:22.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:22 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-13T10:32:22.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:22 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:23.736 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:23 smithi173 ceph-mon[30864]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:23.737 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:23 smithi173 ceph-mon[30864]: Deploying daemon osd.9 on smithi173 2024-09-13T10:32:23.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:23 smithi138 ceph-mon[31033]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:23.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:23 smithi138 ceph-mon[31033]: Deploying daemon osd.9 on smithi173 2024-09-13T10:32:23.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:23 smithi060 ceph-mon[27375]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:23.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:23 smithi060 ceph-mon[27375]: Deploying daemon osd.9 on smithi173 2024-09-13T10:32:25.652 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:25 smithi138 ceph-mon[31033]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:25.679 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:25 smithi173 ceph-mon[30864]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:25.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:25 smithi060 ceph-mon[27375]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:27 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:27 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:27 smithi173 ceph-mon[30864]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:27 smithi173 ceph-mon[30864]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T10:32:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:27.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:27 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:27.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:27 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:27.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:27 smithi138 ceph-mon[31033]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:27.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:27 smithi138 ceph-mon[31033]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T10:32:27.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:27 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:27.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:27 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:27.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:27 smithi060 ceph-mon[27375]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:27.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:27 smithi060 ceph-mon[27375]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T10:32:27.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:28.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:28 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:28.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:28 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:28.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:28 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:28.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:28 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:28.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:28 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:28.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:28 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:28.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:28 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:28 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:28.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:28 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:29 smithi138 ceph-mon[31033]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:29.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:29 smithi060 ceph-mon[27375]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:29.887 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:29 smithi173 ceph-mon[30864]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:31 smithi173 ceph-mon[30864]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:31.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:31 smithi138 ceph-mon[31033]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:31 smithi060 ceph-mon[27375]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:31.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:31.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:31.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:31.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.112 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 9 on host 'smithi173' 2024-09-13T10:32:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:33 smithi173 ceph-mon[30864]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:33 smithi173 ceph-mon[30864]: from='osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-13T10:32:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:33 smithi173 ceph-mon[30864]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-13T10:32:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:33 smithi138 ceph-mon[31033]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:33 smithi138 ceph-mon[31033]: from='osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-13T10:32:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:33 smithi138 ceph-mon[31033]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-13T10:32:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:33 smithi060 ceph-mon[27375]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:33.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:33 smithi060 ceph-mon[27375]: from='osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-13T10:32:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:33 smithi060 ceph-mon[27375]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-13T10:32:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:33.755 DEBUG:teuthology.orchestra.run.smithi173:osd.9> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.9.service 2024-09-13T10:32:33.758 INFO:tasks.cephadm:Deploying osd.10 on smithi173 with /dev/vg_nvme/lv_2... 2024-09-13T10:32:33.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-13T10:32:34.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-mon[30864]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T10:32:34.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-mon[30864]: osdmap e67: 10 total, 9 up, 10 in 2024-09-13T10:32:34.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:34.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-mon[30864]: from='osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:34.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-mon[30864]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:34 smithi138 ceph-mon[31033]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T10:32:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:34 smithi138 ceph-mon[31033]: osdmap e67: 10 total, 9 up, 10 in 2024-09-13T10:32:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:34 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:34 smithi138 ceph-mon[31033]: from='osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:34 smithi138 ceph-mon[31033]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:34.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:34 smithi060 ceph-mon[27375]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T10:32:34.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:34 smithi060 ceph-mon[27375]: osdmap e67: 10 total, 9 up, 10 in 2024-09-13T10:32:34.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:34 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:34.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:34 smithi060 ceph-mon[27375]: from='osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:34.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:34 smithi060 ceph-mon[27375]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:32:34.918 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:32:34.619+0000 7f4106e6a640 -1 osd.9 0 waiting for initial osdmap 2024-09-13T10:32:34.918 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:32:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:32:34.627+0000 7f40fec89640 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:32:35.425 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:32:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:35 smithi173 ceph-mon[30864]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:35 smithi173 ceph-mon[30864]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:32:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:35 smithi173 ceph-mon[30864]: osdmap e68: 10 total, 9 up, 10 in 2024-09-13T10:32:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:35 smithi173 ceph-mon[30864]: from='osd.9 ' entity='osd.9' 2024-09-13T10:32:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:35 smithi138 ceph-mon[31033]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:35 smithi138 ceph-mon[31033]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:32:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:35 smithi138 ceph-mon[31033]: osdmap e68: 10 total, 9 up, 10 in 2024-09-13T10:32:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:35 smithi138 ceph-mon[31033]: from='osd.9 ' entity='osd.9' 2024-09-13T10:32:35.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:35 smithi060 ceph-mon[27375]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:35.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:35 smithi060 ceph-mon[27375]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:32:35.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:35 smithi060 ceph-mon[27375]: osdmap e68: 10 total, 9 up, 10 in 2024-09-13T10:32:35.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:35.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:35.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:35 smithi060 ceph-mon[27375]: from='osd.9 ' entity='osd.9' 2024-09-13T10:32:36.459 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:36 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:36.459 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:36 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:36.459 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:36.459 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:36 smithi173 ceph-mon[30864]: osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706] boot 2024-09-13T10:32:36.459 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:36 smithi173 ceph-mon[30864]: osdmap e69: 10 total, 10 up, 10 in 2024-09-13T10:32:36.459 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:36 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:36 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:36 smithi138 ceph-mon[31033]: osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706] boot 2024-09-13T10:32:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:36 smithi138 ceph-mon[31033]: osdmap e69: 10 total, 10 up, 10 in 2024-09-13T10:32:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:36.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:36 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:36.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:36 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:36 smithi060 ceph-mon[27375]: osd.9 [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706] boot 2024-09-13T10:32:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:36 smithi060 ceph-mon[27375]: osdmap e69: 10 total, 10 up, 10 in 2024-09-13T10:32:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-13T10:32:37.911 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: osdmap e70: 10 total, 10 up, 10 in 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:37.912 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:37.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: osdmap e70: 10 total, 10 up, 10 in 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:37.981 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: osdmap e70: 10 total, 10 up, 10 in 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:38.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:38 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:38 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:38 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi173 2024-09-13T10:32:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:38 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi173 to 10592M 2024-09-13T10:32:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:38 smithi173 ceph-mon[30864]: osdmap e71: 10 total, 10 up, 10 in 2024-09-13T10:32:38.792 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:32:38.826 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-09-13T10:32:39.077 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:32:39.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:38 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:39.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:38 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:39.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:38 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi173 2024-09-13T10:32:39.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:38 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi173 to 10592M 2024-09-13T10:32:39.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:38 smithi138 ceph-mon[31033]: osdmap e71: 10 total, 10 up, 10 in 2024-09-13T10:32:39.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:38 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:39.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:38 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:39.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:38 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi173 2024-09-13T10:32:39.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:38 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi173 to 10592M 2024-09-13T10:32:39.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:38 smithi060 ceph-mon[27375]: osdmap e71: 10 total, 10 up, 10 in 2024-09-13T10:32:39.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:39 smithi173 ceph-mon[30864]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:40.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:39 smithi138 ceph-mon[31033]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:40.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:39 smithi060 ceph-mon[27375]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:40.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:40 smithi173 ceph-mon[30864]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:32:40.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:40 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:32:40.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:40 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:32:40.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:40 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:41.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:40 smithi138 ceph-mon[31033]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:32:41.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:40 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:32:41.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:40 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:32:41.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:40 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:41.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:40 smithi060 ceph-mon[27375]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:32:41.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:40 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:32:41.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:40 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:32:41.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:40 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:41.912 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:41 smithi173 ceph-mon[30864]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:42.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:41 smithi138 ceph-mon[31033]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:42.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:41 smithi060 ceph-mon[27375]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:42.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:42.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:42.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: osdmap e72: 10 total, 10 up, 10 in 2024-09-13T10:32:42.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/1694185587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"} : dispatch 2024-09-13T10:32:42.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"} : dispatch 2024-09-13T10:32:42.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"}]': finished 2024-09-13T10:32:42.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: osdmap e73: 11 total, 10 up, 11 in 2024-09-13T10:32:42.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:32:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: osdmap e72: 10 total, 10 up, 10 in 2024-09-13T10:32:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/1694185587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"} : dispatch 2024-09-13T10:32:43.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"} : dispatch 2024-09-13T10:32:43.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"}]': finished 2024-09-13T10:32:43.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: osdmap e73: 11 total, 10 up, 11 in 2024-09-13T10:32:43.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:32:43.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:43.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: osdmap e72: 10 total, 10 up, 10 in 2024-09-13T10:32:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/1694185587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"} : dispatch 2024-09-13T10:32:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"} : dispatch 2024-09-13T10:32:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d"}]': finished 2024-09-13T10:32:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: osdmap e73: 11 total, 10 up, 11 in 2024-09-13T10:32:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:32:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:43 smithi173 ceph-mon[30864]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:43 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/3226658968' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:32:44.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:43 smithi138 ceph-mon[31033]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:44.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:43 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/3226658968' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:32:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:43 smithi060 ceph-mon[27375]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:44.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:43 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/3226658968' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:32:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:45 smithi173 ceph-mon[30864]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:46.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:45 smithi138 ceph-mon[31033]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:46.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:45 smithi060 ceph-mon[27375]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:47.978 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:47 smithi173 ceph-mon[30864]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:48.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:47 smithi138 ceph-mon[31033]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:48.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:47 smithi060 ceph-mon[27375]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:49.860 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:49 smithi173 ceph-mon[30864]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:50.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:49 smithi138 ceph-mon[31033]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:50.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:49 smithi060 ceph-mon[27375]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:50.709 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:50 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-13T10:32:50.709 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:50 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:51.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:50 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-13T10:32:51.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:50 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:51.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:50 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-13T10:32:51.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:50 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:51.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:51 smithi173 ceph-mon[30864]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:51.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:51 smithi173 ceph-mon[30864]: Deploying daemon osd.10 on smithi173 2024-09-13T10:32:52.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:51 smithi138 ceph-mon[31033]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:52.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:51 smithi138 ceph-mon[31033]: Deploying daemon osd.10 on smithi173 2024-09-13T10:32:52.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:51 smithi060 ceph-mon[27375]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:52.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:51 smithi060 ceph-mon[27375]: Deploying daemon osd.10 on smithi173 2024-09-13T10:32:52.973 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:52 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:32:52.973 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:52 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:32:53.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:52 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:32:53.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:52 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:32:53.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:52 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:32:53.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:52 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:32:53.870 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:53 smithi173 ceph-mon[30864]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:53.871 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:53 smithi173 ceph-mon[30864]: osdmap e74: 11 total, 10 up, 11 in 2024-09-13T10:32:53.871 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:53 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:32:54.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:53 smithi138 ceph-mon[31033]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:54.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:53 smithi138 ceph-mon[31033]: osdmap e74: 11 total, 10 up, 11 in 2024-09-13T10:32:54.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:53 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:32:54.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:53 smithi060 ceph-mon[27375]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:53 smithi060 ceph-mon[27375]: osdmap e74: 11 total, 10 up, 11 in 2024-09-13T10:32:54.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:53 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:32:56.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:55 smithi173 ceph-mon[30864]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:56.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:55 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:56.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:55 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:56.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:55 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:56.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:55 smithi138 ceph-mon[31033]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:56.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:55 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:56.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:55 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:56.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:55 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:56.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:55 smithi060 ceph-mon[27375]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:56.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:55 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:32:56.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:55 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:56.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:55 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:57.918 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:57 smithi173 ceph-mon[30864]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:58.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:57 smithi138 ceph-mon[31033]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:58.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:57 smithi060 ceph-mon[27375]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:59 smithi173 ceph-mon[30864]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:59.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:32:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:59 smithi138 ceph-mon[31033]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:59.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:32:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:59 smithi060 ceph-mon[27375]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:32:59.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:32:59.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:32:59.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:32:59.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:32:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.217 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 10 on host 'smithi173' 2024-09-13T10:33:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:01 smithi173 ceph-mon[30864]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:33:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:01 smithi173 ceph-mon[30864]: from='osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-13T10:33:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:01 smithi173 ceph-mon[30864]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-13T10:33:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:01 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:01 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:01 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:01 smithi138 ceph-mon[31033]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:33:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:01 smithi138 ceph-mon[31033]: from='osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-13T10:33:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:01 smithi138 ceph-mon[31033]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-13T10:33:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:01 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:01 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:01 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:01 smithi060 ceph-mon[27375]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:33:01.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:01 smithi060 ceph-mon[27375]: from='osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-13T10:33:01.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:01 smithi060 ceph-mon[27375]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-13T10:33:01.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:01 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:01.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:01 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:01 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:01.915 DEBUG:teuthology.orchestra.run.smithi173:osd.10> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.10.service 2024-09-13T10:33:01.917 INFO:tasks.cephadm:Deploying osd.11 on smithi173 with /dev/vg_nvme/lv_1... 2024-09-13T10:33:01.918 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-13T10:33:02.488 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-mon[30864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T10:33:02.489 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-mon[30864]: osdmap e75: 11 total, 10 up, 11 in 2024-09-13T10:33:02.489 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:02.489 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-mon[30864]: from='osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:02.489 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-mon[30864]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:02.489 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:33:02.239+0000 7f2d723fa640 -1 osd.10 0 waiting for initial osdmap 2024-09-13T10:33:02.489 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:33:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:33:02.246+0000 7f2d6e221640 -1 osd.10 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:33:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:02 smithi138 ceph-mon[31033]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T10:33:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:02 smithi138 ceph-mon[31033]: osdmap e75: 11 total, 10 up, 11 in 2024-09-13T10:33:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:02 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:02 smithi138 ceph-mon[31033]: from='osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:02 smithi138 ceph-mon[31033]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:02.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:02 smithi060 ceph-mon[27375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T10:33:02.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:02 smithi060 ceph-mon[27375]: osdmap e75: 11 total, 10 up, 11 in 2024-09-13T10:33:02.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:02 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:02.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:02 smithi060 ceph-mon[27375]: from='osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:02.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:02 smithi060 ceph-mon[27375]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:02.712 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:33:03.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:03 smithi060 ceph-mon[27375]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:33:03.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:03 smithi060 ceph-mon[27375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:33:03.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:03 smithi060 ceph-mon[27375]: osdmap e76: 11 total, 10 up, 11 in 2024-09-13T10:33:03.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:03.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:03 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:03 smithi173 ceph-mon[30864]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:33:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:03 smithi173 ceph-mon[30864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:33:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:03 smithi173 ceph-mon[30864]: osdmap e76: 11 total, 10 up, 11 in 2024-09-13T10:33:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:03 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:03 smithi138 ceph-mon[31033]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 275 MiB used, 894 GiB / 894 GiB avail 2024-09-13T10:33:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:03 smithi138 ceph-mon[31033]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:33:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:03 smithi138 ceph-mon[31033]: osdmap e76: 11 total, 10 up, 11 in 2024-09-13T10:33:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:03 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:04 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:04 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:04 smithi173 ceph-mon[30864]: osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063] boot 2024-09-13T10:33:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:04 smithi173 ceph-mon[30864]: osdmap e77: 11 total, 11 up, 11 in 2024-09-13T10:33:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:04 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:04 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:04 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:04 smithi138 ceph-mon[31033]: osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063] boot 2024-09-13T10:33:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:04 smithi138 ceph-mon[31033]: osdmap e77: 11 total, 11 up, 11 in 2024-09-13T10:33:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:04 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:04.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:04 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:04.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:04 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:04 smithi060 ceph-mon[27375]: osd.10 [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063] boot 2024-09-13T10:33:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:04 smithi060 ceph-mon[27375]: osdmap e77: 11 total, 11 up, 11 in 2024-09-13T10:33:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:04 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-13T10:33:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:05 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:05 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:05 smithi173 ceph-mon[30864]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:05.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:05 smithi173 ceph-mon[30864]: osdmap e78: 11 total, 11 up, 11 in 2024-09-13T10:33:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:05 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:05 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:05 smithi138 ceph-mon[31033]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:05 smithi138 ceph-mon[31033]: osdmap e78: 11 total, 11 up, 11 in 2024-09-13T10:33:05.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:05 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:05.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:05 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:05.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:05 smithi060 ceph-mon[27375]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:05.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:05 smithi060 ceph-mon[27375]: osdmap e78: 11 total, 11 up, 11 in 2024-09-13T10:33:06.265 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {auth=21,config=2,health=12,kv=60,logm=5} crc {auth=328217058,config=1548763834,health=3897446751,kv=2434016570,logm=702364389}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=5377860}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=630542922}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3535445167}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=660159139}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3925274869}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2951151600}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=1347278780}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3437828447}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2675942596}) 2024-09-13T10:33:06.266 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3153656280}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3578843450}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3882083807}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2129432747}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=816387468}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=1928403998}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {logm=35,mdsmap=3,mgr=19,mgr_command_descs=1,mgr_metadata=2,mgrstat=13,mon_config_key=27} crc {logm=439529871,mdsmap=1655132698,mgr=831520534,mgr_command_descs=899306875,mgr_metadata=4195947585,mgrstat=35677085,mon_config_key=999470227}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=100} crc {mon_config_key=130609982}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=100} crc {mon_config_key=535375088}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=43,monmap=5,nvmeofgw=3,osd_metadata=11,osd_pg_creating=1,osdmap=37} crc {mon_config_key=509373030,monmap=2694462162,nvmeofgw=598337686,osd_metadata=1819665884,osd_pg_creating=2290527312,osdmap=1485342842}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: scrub ok on 0,1,2: ScrubResult(keys {osdmap=78} crc {osdmap=2756130791}) 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.267 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:06.268 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:06 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {auth=21,config=2,health=12,kv=60,logm=5} crc {auth=328217058,config=1548763834,health=3897446751,kv=2434016570,logm=702364389}) 2024-09-13T10:33:06.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=5377860}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=630542922}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3535445167}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=660159139}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3925274869}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2951151600}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=1347278780}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3437828447}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2675942596}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3153656280}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3578843450}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3882083807}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2129432747}) 2024-09-13T10:33:06.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=816387468}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=1928403998}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {logm=35,mdsmap=3,mgr=19,mgr_command_descs=1,mgr_metadata=2,mgrstat=13,mon_config_key=27} crc {logm=439529871,mdsmap=1655132698,mgr=831520534,mgr_command_descs=899306875,mgr_metadata=4195947585,mgrstat=35677085,mon_config_key=999470227}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=100} crc {mon_config_key=130609982}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=100} crc {mon_config_key=535375088}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=43,monmap=5,nvmeofgw=3,osd_metadata=11,osd_pg_creating=1,osdmap=37} crc {mon_config_key=509373030,monmap=2694462162,nvmeofgw=598337686,osd_metadata=1819665884,osd_pg_creating=2290527312,osdmap=1485342842}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: scrub ok on 0,1,2: ScrubResult(keys {osdmap=78} crc {osdmap=2756130791}) 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:06.603 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:06.603 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:06 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {auth=21,config=2,health=12,kv=60,logm=5} crc {auth=328217058,config=1548763834,health=3897446751,kv=2434016570,logm=702364389}) 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=5377860}) 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=630542922}) 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3535445167}) 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=660159139}) 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3925274869}) 2024-09-13T10:33:06.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2951151600}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=1347278780}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3437828447}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2675942596}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3153656280}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3578843450}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=3882083807}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=2129432747}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=816387468}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=100} crc {logm=1928403998}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {logm=35,mdsmap=3,mgr=19,mgr_command_descs=1,mgr_metadata=2,mgrstat=13,mon_config_key=27} crc {logm=439529871,mdsmap=1655132698,mgr=831520534,mgr_command_descs=899306875,mgr_metadata=4195947585,mgrstat=35677085,mon_config_key=999470227}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=100} crc {mon_config_key=130609982}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=100} crc {mon_config_key=535375088}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {mon_config_key=43,monmap=5,nvmeofgw=3,osd_metadata=11,osd_pg_creating=1,osdmap=37} crc {mon_config_key=509373030,monmap=2694462162,nvmeofgw=598337686,osd_metadata=1819665884,osd_pg_creating=2290527312,osdmap=1485342842}) 2024-09-13T10:33:06.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: scrub ok on 0,1,2: ScrubResult(keys {osdmap=78} crc {osdmap=2756130791}) 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:06.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:06 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:07 smithi173 ceph-mon[30864]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:07 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi173 2024-09-13T10:33:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:07 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi173 to 7061M 2024-09-13T10:33:07.542 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:33:07.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-09-13T10:33:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:07 smithi138 ceph-mon[31033]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:07 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi173 2024-09-13T10:33:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:07 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi173 to 7061M 2024-09-13T10:33:07.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:07 smithi060 ceph-mon[27375]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:07.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:07 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi173 2024-09-13T10:33:07.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:07 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi173 to 7061M 2024-09-13T10:33:07.823 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:33:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:09 smithi173 ceph-mon[30864]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:09 smithi173 ceph-mon[30864]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:33:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:33:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:33:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:09 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:09 smithi138 ceph-mon[31033]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:09 smithi138 ceph-mon[31033]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:33:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:33:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:33:09.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:09 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:09 smithi060 ceph-mon[27375]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:09 smithi060 ceph-mon[27375]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T10:33:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-13T10:33:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-13T10:33:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:09 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/1948811693' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"} : dispatch 2024-09-13T10:33:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"} : dispatch 2024-09-13T10:33:11.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"}]': finished 2024-09-13T10:33:11.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T10:33:11.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:11 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:11.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:11.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:11.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:11.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/1948811693' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"} : dispatch 2024-09-13T10:33:11.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"} : dispatch 2024-09-13T10:33:11.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"}]': finished 2024-09-13T10:33:11.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T10:33:11.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:11 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:11.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:11.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/1948811693' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"} : dispatch 2024-09-13T10:33:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"} : dispatch 2024-09-13T10:33:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362"}]': finished 2024-09-13T10:33:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T10:33:11.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:11 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:12.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:12 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:33:12.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:12 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:33:12.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:12 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/3912150045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:33:12.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:12 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:33:12.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:12 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:33:12.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:12 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/3912150045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:33:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:12 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:33:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:12 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:33:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:12 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/3912150045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-13T10:33:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:13 smithi173 ceph-mon[30864]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:13 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:13 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:13.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:13 smithi138 ceph-mon[31033]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:13.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:13 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:13.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:13 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:13.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:13 smithi060 ceph-mon[27375]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:13.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:13 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:13.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:13 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:14.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:14 smithi138 ceph-mon[31033]: osdmap e80: 12 total, 11 up, 12 in 2024-09-13T10:33:14.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:14 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:14.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:14 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:14.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:14 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:14.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:14 smithi060 ceph-mon[27375]: osdmap e80: 12 total, 11 up, 12 in 2024-09-13T10:33:14.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:14 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:14.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:14 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:14.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:14 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:14.675 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:14 smithi173 ceph-mon[30864]: osdmap e80: 12 total, 11 up, 12 in 2024-09-13T10:33:14.676 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:14 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:14.676 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:14 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:14.676 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:14 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:15.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:15 smithi138 ceph-mon[31033]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:15.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:15 smithi138 ceph-mon[31033]: osdmap e81: 12 total, 11 up, 12 in 2024-09-13T10:33:15.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:15 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:15.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:15 smithi060 ceph-mon[27375]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:15.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:15 smithi060 ceph-mon[27375]: osdmap e81: 12 total, 11 up, 12 in 2024-09-13T10:33:15.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:15 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:15.726 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:15 smithi173 ceph-mon[30864]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:15.726 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:15 smithi173 ceph-mon[30864]: osdmap e81: 12 total, 11 up, 12 in 2024-09-13T10:33:15.726 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:15 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:17.418 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:17 smithi173 ceph-mon[30864]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:17.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:17 smithi138 ceph-mon[31033]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:17.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:17 smithi060 ceph-mon[27375]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:19 smithi173 ceph-mon[30864]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:19 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-13T10:33:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:19 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:19 smithi173 ceph-mon[30864]: Deploying daemon osd.11 on smithi173 2024-09-13T10:33:19.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:19 smithi138 ceph-mon[31033]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:19.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:19 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-13T10:33:19.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:19 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:19.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:19 smithi138 ceph-mon[31033]: Deploying daemon osd.11 on smithi173 2024-09-13T10:33:19.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:19 smithi060 ceph-mon[27375]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:19.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:19 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-13T10:33:19.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:19 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:19.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:19 smithi060 ceph-mon[27375]: Deploying daemon osd.11 on smithi173 2024-09-13T10:33:21.417 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:21 smithi173 ceph-mon[30864]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:21.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:21 smithi138 ceph-mon[31033]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:21.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:21 smithi060 ceph-mon[27375]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:23.395 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:23 smithi173 ceph-mon[30864]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:23.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:23 smithi138 ceph-mon[31033]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:23.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:23 smithi060 ceph-mon[27375]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:24.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:24.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:24.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:24.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:24.668 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:24.668 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:24.668 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:25.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:25 smithi138 ceph-mon[31033]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:25.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:25 smithi060 ceph-mon[27375]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:25.633 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:25 smithi173 ceph-mon[30864]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:27.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:27 smithi138 ceph-mon[31033]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:27.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:27.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:27.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:27 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:27 smithi060 ceph-mon[27375]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:27.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:27.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:27.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:27 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:27 smithi173 ceph-mon[30864]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:27.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:27 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.056 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 11 on host 'smithi173' 2024-09-13T10:33:29.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:29 smithi138 ceph-mon[31033]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:29.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:29 smithi138 ceph-mon[31033]: from='osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-13T10:33:29.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:29 smithi138 ceph-mon[31033]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-13T10:33:29.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:29.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:29 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:29 smithi060 ceph-mon[27375]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:29.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:29 smithi060 ceph-mon[27375]: from='osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-13T10:33:29.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:29 smithi060 ceph-mon[27375]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-13T10:33:29.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:29.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:29 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.652 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:29 smithi173 ceph-mon[30864]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:29.652 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:29 smithi173 ceph-mon[30864]: from='osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-13T10:33:29.653 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:29 smithi173 ceph-mon[30864]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-13T10:33:29.653 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:33:29.653 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.653 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:29 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:29.684 DEBUG:teuthology.orchestra.run.smithi173:osd.11> sudo journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.11.service 2024-09-13T10:33:29.686 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-13T10:33:29.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T10:33:29.954 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:30 smithi060 ceph-mon[27375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T10:33:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:30 smithi060 ceph-mon[27375]: osdmap e82: 12 total, 11 up, 12 in 2024-09-13T10:33:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:30 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:30 smithi060 ceph-mon[27375]: from='osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:30 smithi060 ceph-mon[27375]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:30 smithi138 ceph-mon[31033]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T10:33:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:30 smithi138 ceph-mon[31033]: osdmap e82: 12 total, 11 up, 12 in 2024-09-13T10:33:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:30 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:30 smithi138 ceph-mon[31033]: from='osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:30 smithi138 ceph-mon[31033]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-mon[30864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T10:33:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-mon[30864]: osdmap e82: 12 total, 11 up, 12 in 2024-09-13T10:33:30.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:30.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-mon[30864]: from='osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:30.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-mon[30864]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]} : dispatch 2024-09-13T10:33:30.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:33:30.329+0000 7fd506ed6640 -1 osd.11 0 waiting for initial osdmap 2024-09-13T10:33:30.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:33:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:33:30.335+0000 7fd502cfd640 -1 osd.11 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:33:31.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: osdmap e83: 12 total, 11 up, 12 in 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:31 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2063214542' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T10:33:31.451 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":83,"num_osds":12,"num_up_osds":11,"osd_up_since":1726223583,"num_in_osds":12,"osd_in_since":1726223591,"num_remapped_pgs":0} 2024-09-13T10:33:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:33:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: osdmap e83: 12 total, 11 up, 12 in 2024-09-13T10:33:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:31.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:31.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:31 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2063214542' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T10:33:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 983 GiB / 983 GiB avail 2024-09-13T10:33:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-09-13T10:33:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: osdmap e83: 12 total, 11 up, 12 in 2024-09-13T10:33:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:31 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2063214542' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T10:33:32.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T10:33:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:32 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:32 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:32 smithi173 ceph-mon[30864]: osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497] boot 2024-09-13T10:33:32.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:32 smithi173 ceph-mon[30864]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T10:33:32.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:32 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:32.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:32 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:32.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:32 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:32.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:32 smithi138 ceph-mon[31033]: osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497] boot 2024-09-13T10:33:32.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:32 smithi138 ceph-mon[31033]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T10:33:32.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:32 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:32.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:32 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:32.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:32 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:32.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:32 smithi060 ceph-mon[27375]: osd.11 [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497] boot 2024-09-13T10:33:32.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:32 smithi060 ceph-mon[27375]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T10:33:32.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:32 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-13T10:33:32.728 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T10:33:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: Detected new or changed devices on smithi173 2024-09-13T10:33:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: Adjusting osd_memory_target on smithi173 to 5296M 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:33.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:33 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T10:33:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: Detected new or changed devices on smithi173 2024-09-13T10:33:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: Adjusting osd_memory_target on smithi173 to 5296M 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:33 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T10:33:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: Detected new or changed devices on smithi173 2024-09-13T10:33:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: Adjusting osd_memory_target on smithi173 to 5296M 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:33:33.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:33 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:33:34.014 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"num_osds":12,"num_up_osds":12,"osd_up_since":1726223611,"num_in_osds":12,"osd_in_since":1726223591,"num_remapped_pgs":0} 2024-09-13T10:33:34.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T10:33:34.268 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:34 smithi138 ceph-mon[31033]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T10:33:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:34 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2113710852' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T10:33:34.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:34 smithi060 ceph-mon[27375]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T10:33:34.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:34 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2113710852' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T10:33:34.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:34 smithi173 ceph-mon[30864]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T10:33:34.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:34 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2113710852' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-13T10:33:34.924 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:34.924 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","created":"2024-09-13T10:26:24.281921+0000","modified":"2024-09-13T10:33:33.328547+0000","last_up_change":"2024-09-13T10:33:31.317557+0000","last_in_change":"2024-09-13T10:33:11.111853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T10:29:06.355752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3db36d02-fea0-4b18-8994-97158486c7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6803","nonce":756062006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6805","nonce":756062006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6809","nonce":756062006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6807","nonce":756062006}]},"public_addr":"172.21.15.60:6803/756062006","cluster_addr":"172.21.15.60:6805/756062006","heartbeat_back_addr":"172.21.15.60:6809/756062006","heartbeat_front_addr":"172.21.15.60:6807/756062006","state":["exists","up"]},{"osd":1,"uuid":"f2323c13-004c-47e1-81ed-44e6c9fb193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6811","nonce":3814817161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6813","nonce":3814817161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6817","nonce":3814817161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6815","nonce":3814817161}]},"public_addr":"172.21.15.60:6811/3814817161","cluster_addr":"172.21.15.60:6813/3814817161","heartbeat_back_addr":"172.21.15.60:6817/3814817161","heartbeat_front_addr":"172.21.15.60:6815/3814817161","state":["exists","up"]},{"osd":2,"uuid":"bda5f20f-8413-4a97-9969-1eea2f1f7b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6819","nonce":1235426811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6821","nonce":1235426811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6825","nonce":1235426811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6823","nonce":1235426811}]},"public_addr":"172.21.15.60:6819/1235426811","cluster_addr":"172.21.15.60:6821/1235426811","heartbeat_back_addr":"172.21.15.60:6825/1235426811","heartbeat_front_addr":"172.21.15.60:6823/1235426811","state":["exists","up"]},{"osd":3,"uuid":"43cba2dc-ab85-4d9a-9b2e-c4fd527077a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6827","nonce":3263875168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6829","nonce":3263875168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6833","nonce":3263875168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6831","nonce":3263875168}]},"public_addr":"172.21.15.60:6827/3263875168","cluster_addr":"172.21.15.60:6829/3263875168","heartbeat_back_addr":"172.21.15.60:6833/3263875168","heartbeat_front_addr":"172.21.15.60:6831/3263875168","state":["exists","up"]},{"osd":4,"uuid":"b76daf81-be0c-42b5-b262-a4a0cfd615f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6801","nonce":2143931671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2143931671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2143931671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2143931671}]},"public_addr":"172.21.15.138:6801/2143931671","cluster_addr":"172.21.15.138:6803/2143931671","heartbeat_back_addr":"172.21.15.138:6807/2143931671","heartbeat_front_addr":"172.21.15.138:6805/2143931671","state":["exists","up"]},{"osd":5,"uuid":"9037c1e7-d476-4980-8155-6244430c9011","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6809","nonce":2632797914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6811","nonce":2632797914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6815","nonce":2632797914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6813","nonce":2632797914}]},"public_addr":"172.21.15.138:6809/2632797914","cluster_addr":"172.21.15.138:6811/2632797914","heartbeat_back_addr":"172.21.15.138:6815/2632797914","heartbeat_front_addr":"172.21.15.138:6813/2632797914","state":["exists","up"]},{"osd":6,"uuid":"b635470b-4619-4a8a-b016-1811d5049544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6817","nonce":3026996312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6819","nonce":3026996312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6823","nonce":3026996312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6821","nonce":3026996312}]},"public_addr":"172.21.15.138:6817/3026996312","cluster_addr":"172.21.15.138:6819/3026996312","heartbeat_back_addr":"172.21.15.138:6823/3026996312","heartbeat_front_addr":"172.21.15.138:6821/3026996312","state":["exists","up"]},{"osd":7,"uuid":"4c58cdb7-9705-4a68-816f-8d6ddeca34d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6825","nonce":1381162744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6827","nonce":1381162744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6831","nonce":1381162744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6829","nonce":1381162744}]},"public_addr":"172.21.15.138:6825/1381162744","cluster_addr":"172.21.15.138:6827/1381162744","heartbeat_back_addr":"172.21.15.138:6831/1381162744","heartbeat_front_addr":"172.21.15.138:6829/1381162744","state":["exists","up"]},{"osd":8,"uuid":"b77d95ef-5b65-4446-ae22-ef2dfe094667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6801","nonce":1825049558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6803","nonce":1825049558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6807","nonce":1825049558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6805","nonce":1825049558}]},"public_addr":"172.21.15.173:6801/1825049558","cluster_addr":"172.21.15.173:6803/1825049558","heartbeat_back_addr":"172.21.15.173:6807/1825049558","heartbeat_front_addr":"172.21.15.173:6805/1825049558","state":["exists","up"]},{"osd":9,"uuid":"b89d58eb-56f0-4ede-aa1b-8d6c25be30af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6809","nonce":417733706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6811","nonce":417733706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6815","nonce":417733706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6813","nonce":417733706}]},"public_addr":"172.21.15.173:6809/417733706","cluster_addr":"172.21.15.173:6811/417733706","heartbeat_back_addr":"172.21.15.173:6815/417733706","heartbeat_front_addr":"172.21.15.173:6813/417733706","state":["exists","up"]},{"osd":10,"uuid":"46ddc07a-0a77-445e-bc19-a345e2cdce5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6817","nonce":3542783063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6819","nonce":3542783063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6823","nonce":3542783063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6821","nonce":3542783063}]},"public_addr":"172.21.15.173:6817/3542783063","cluster_addr":"172.21.15.173:6819/3542783063","heartbeat_back_addr":"172.21.15.173:6823/3542783063","heartbeat_front_addr":"172.21.15.173:6821/3542783063","state":["exists","up"]},{"osd":11,"uuid":"6476cc59-b51a-43a8-962e-1ae48119c362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6825","nonce":4131510497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6827","nonce":4131510497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6831","nonce":4131510497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6829","nonce":4131510497}]},"public_addr":"172.21.15.173:6825/4131510497","cluster_addr":"172.21.15.173:6827/4131510497","heartbeat_back_addr":"172.21.15.173:6831/4131510497","heartbeat_front_addr":"172.21.15.173:6829/4131510497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:25.483430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:36.760500+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:03.630608+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:30.349970+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:10.754534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:13.998635+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:41.232312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:51.907686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:12.388490+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:33.334404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:01.420385+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:29.765854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/2156160235":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1488046566":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6801/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6800/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/2405347332":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1860079602":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6801/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6800/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/2669724212":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/1079154737":"2024-09-14T10:26:47.425146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:33:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:35 smithi138 ceph-mon[31033]: pgmap v267: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:35 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2190404234' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:35.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:35 smithi060 ceph-mon[27375]: pgmap v267: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:35.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:35 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2190404234' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:35.675 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:29:06.355752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 11.760000228881836, 'score_stable': 11.760000228881836, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T10:33:35.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-13T10:33:35.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:35 smithi173 ceph-mon[30864]: pgmap v267: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:35.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:35 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2190404234' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:35.930 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:36.572 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-13T10:33:37.260 INFO:tasks.cephadm:Setting up client nodes... 2024-09-13T10:33:37.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T10:33:37.520 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:37 smithi138 ceph-mon[31033]: pgmap v268: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:37 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2625458202' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-13T10:33:37.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:37 smithi060 ceph-mon[27375]: pgmap v268: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:37.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:37 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2625458202' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-13T10:33:37.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:37 smithi173 ceph-mon[30864]: pgmap v268: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:37.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:37 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2625458202' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-13T10:33:38.274 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-13T10:33:38.274 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQACFeRmrSMqEBAA8T8XCA5J57PYbT9Ood1CtQ== 2024-09-13T10:33:38.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:38 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:38.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:38 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:38.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:38 smithi060 ceph-mon[27375]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T10:33:38.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:38 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1622110310' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:38.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:38 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1622110310' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:38 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:38 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:38 smithi173 ceph-mon[30864]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T10:33:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:38 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1622110310' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:38.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:38 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1622110310' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:38 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:38.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:38 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:38.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:38 smithi138 ceph-mon[31033]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T10:33:38.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:38 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1622110310' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:38.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:38 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1622110310' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:39.083 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T10:33:39.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-13T10:33:39.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T10:33:39.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T10:33:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:33:39.501 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:39 smithi138 ceph-mon[31033]: pgmap v270: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:39.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:39 smithi060 ceph-mon[27375]: pgmap v270: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:39.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:39 smithi173 ceph-mon[30864]: pgmap v270: 1 pgs: 1 activating; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2/4 objects degraded (50.000%) 2024-09-13T10:33:40.224 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-09-13T10:33:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAEFeRmgfAIDRAA1BERjj/HVp61UN1iXMgR5Q== 2024-09-13T10:33:40.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:40 smithi138 ceph-mon[31033]: from='client.? 172.21.15.138:0/1282173261' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:40.618 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:40 smithi138 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:40.618 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:40 smithi138 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:40.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:40 smithi060 ceph-mon[27375]: from='client.? 172.21.15.138:0/1282173261' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:40.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:40 smithi060 ceph-mon[27375]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:40.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:40 smithi060 ceph-mon[27375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:40.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:40 smithi173 ceph-mon[30864]: from='client.? 172.21.15.138:0/1282173261' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:40.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:40 smithi173 ceph-mon[30864]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:40.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:40 smithi173 ceph-mon[30864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:40.984 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-13T10:33:40.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-13T10:33:40.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T10:33:41.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T10:33:41.281 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:33:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:41 smithi138 ceph-mon[31033]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-13T10:33:41.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:41 smithi060 ceph-mon[27375]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-13T10:33:41.652 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:41 smithi173 ceph-mon[30864]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-13T10:33:42.001 INFO:teuthology.orchestra.run.smithi173.stdout:[client.2] 2024-09-13T10:33:42.001 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQAFFeRmx+FUOxAAMUZnfTJAPeR9wyyhT4/12g== 2024-09-13T10:33:42.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/1596848328' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:42.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:42 smithi060 ceph-mon[27375]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:42.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:42 smithi060 ceph-mon[27375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:42.682 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:42 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/1596848328' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:42.682 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:42 smithi173 ceph-mon[30864]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:42.682 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:42 smithi173 ceph-mon[30864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:42.713 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-13T10:33:42.713 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-09-13T10:33:42.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-13T10:33:42.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-13T10:33:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:42 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/1596848328' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:42 smithi138 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-13T10:33:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:42 smithi138 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T10:33:43.020 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:43.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:43 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:43.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:43 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:43.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:43 smithi060 ceph-mon[27375]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-13T10:33:43.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:43 smithi060 ceph-mon[27375]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T10:33:43.639 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:43 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:43 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:43 smithi173 ceph-mon[30864]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-13T10:33:43.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:43 smithi173 ceph-mon[30864]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T10:33:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:43 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:43 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:43 smithi138 ceph-mon[31033]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-13T10:33:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:43 smithi138 ceph-mon[31033]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T10:33:44.355 INFO:teuthology.orchestra.run.smithi060.stdout:[{"version":26,"timestamp":"2024-09-13T10:33:32.590872+0000","name":"","changes":[{"name":"osd/host:smithi173/osd_memory_target","previous_value":"7404390673","new_value":"5553293004"}]}] 2024-09-13T10:33:44.355 INFO:tasks.ceph_manager:config epoch is 26 2024-09-13T10:33:44.355 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T10:33:44.356 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-13T10:33:44.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-13T10:33:44.607 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:44.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:44 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/217888239' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-13T10:33:44.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:44 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/217888239' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-13T10:33:44.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:44 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/217888239' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-13T10:33:45.310 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:45.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:45 smithi060 ceph-mon[27375]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:33:45.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:45 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/31453969' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-13T10:33:45.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:45 smithi173 ceph-mon[30864]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:33:45.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:45 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/31453969' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-13T10:33:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:45 smithi138 ceph-mon[31033]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-13T10:33:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:45 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/31453969' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-13T10:33:45.954 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":460427693},{"type":"v1","addr":"172.21.15.60:6801","nonce":460427693}]},"active_addr":"172.21.15.60:6801/460427693","active_change":"2024-09-13T10:27:11.894486+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24104,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.60:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":5245877}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":146157419}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2317117880}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1728008012}]}]} 2024-09-13T10:33:45.961 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-13T10:33:45.961 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-13T10:33:45.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T10:33:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:46.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:46 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:46.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:46 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:46.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:46 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:46.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:46 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:46.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:46 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:46.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:46 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:47.010 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:47.010 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","created":"2024-09-13T10:26:24.281921+0000","modified":"2024-09-13T10:33:46.379238+0000","last_up_change":"2024-09-13T10:33:31.317557+0000","last_in_change":"2024-09-13T10:33:11.111853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T10:29:06.355752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3db36d02-fea0-4b18-8994-97158486c7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6803","nonce":756062006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6805","nonce":756062006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6809","nonce":756062006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6807","nonce":756062006}]},"public_addr":"172.21.15.60:6803/756062006","cluster_addr":"172.21.15.60:6805/756062006","heartbeat_back_addr":"172.21.15.60:6809/756062006","heartbeat_front_addr":"172.21.15.60:6807/756062006","state":["exists","up"]},{"osd":1,"uuid":"f2323c13-004c-47e1-81ed-44e6c9fb193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6811","nonce":3814817161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6813","nonce":3814817161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6817","nonce":3814817161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6815","nonce":3814817161}]},"public_addr":"172.21.15.60:6811/3814817161","cluster_addr":"172.21.15.60:6813/3814817161","heartbeat_back_addr":"172.21.15.60:6817/3814817161","heartbeat_front_addr":"172.21.15.60:6815/3814817161","state":["exists","up"]},{"osd":2,"uuid":"bda5f20f-8413-4a97-9969-1eea2f1f7b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6819","nonce":1235426811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6821","nonce":1235426811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6825","nonce":1235426811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6823","nonce":1235426811}]},"public_addr":"172.21.15.60:6819/1235426811","cluster_addr":"172.21.15.60:6821/1235426811","heartbeat_back_addr":"172.21.15.60:6825/1235426811","heartbeat_front_addr":"172.21.15.60:6823/1235426811","state":["exists","up"]},{"osd":3,"uuid":"43cba2dc-ab85-4d9a-9b2e-c4fd527077a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6827","nonce":3263875168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6829","nonce":3263875168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6833","nonce":3263875168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6831","nonce":3263875168}]},"public_addr":"172.21.15.60:6827/3263875168","cluster_addr":"172.21.15.60:6829/3263875168","heartbeat_back_addr":"172.21.15.60:6833/3263875168","heartbeat_front_addr":"172.21.15.60:6831/3263875168","state":["exists","up"]},{"osd":4,"uuid":"b76daf81-be0c-42b5-b262-a4a0cfd615f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6801","nonce":2143931671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2143931671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2143931671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2143931671}]},"public_addr":"172.21.15.138:6801/2143931671","cluster_addr":"172.21.15.138:6803/2143931671","heartbeat_back_addr":"172.21.15.138:6807/2143931671","heartbeat_front_addr":"172.21.15.138:6805/2143931671","state":["exists","up"]},{"osd":5,"uuid":"9037c1e7-d476-4980-8155-6244430c9011","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6809","nonce":2632797914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6811","nonce":2632797914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6815","nonce":2632797914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6813","nonce":2632797914}]},"public_addr":"172.21.15.138:6809/2632797914","cluster_addr":"172.21.15.138:6811/2632797914","heartbeat_back_addr":"172.21.15.138:6815/2632797914","heartbeat_front_addr":"172.21.15.138:6813/2632797914","state":["exists","up"]},{"osd":6,"uuid":"b635470b-4619-4a8a-b016-1811d5049544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6817","nonce":3026996312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6819","nonce":3026996312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6823","nonce":3026996312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6821","nonce":3026996312}]},"public_addr":"172.21.15.138:6817/3026996312","cluster_addr":"172.21.15.138:6819/3026996312","heartbeat_back_addr":"172.21.15.138:6823/3026996312","heartbeat_front_addr":"172.21.15.138:6821/3026996312","state":["exists","up"]},{"osd":7,"uuid":"4c58cdb7-9705-4a68-816f-8d6ddeca34d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6825","nonce":1381162744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6827","nonce":1381162744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6831","nonce":1381162744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6829","nonce":1381162744}]},"public_addr":"172.21.15.138:6825/1381162744","cluster_addr":"172.21.15.138:6827/1381162744","heartbeat_back_addr":"172.21.15.138:6831/1381162744","heartbeat_front_addr":"172.21.15.138:6829/1381162744","state":["exists","up"]},{"osd":8,"uuid":"b77d95ef-5b65-4446-ae22-ef2dfe094667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6801","nonce":1825049558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6803","nonce":1825049558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6807","nonce":1825049558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6805","nonce":1825049558}]},"public_addr":"172.21.15.173:6801/1825049558","cluster_addr":"172.21.15.173:6803/1825049558","heartbeat_back_addr":"172.21.15.173:6807/1825049558","heartbeat_front_addr":"172.21.15.173:6805/1825049558","state":["exists","up"]},{"osd":9,"uuid":"b89d58eb-56f0-4ede-aa1b-8d6c25be30af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6809","nonce":417733706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6811","nonce":417733706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6815","nonce":417733706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6813","nonce":417733706}]},"public_addr":"172.21.15.173:6809/417733706","cluster_addr":"172.21.15.173:6811/417733706","heartbeat_back_addr":"172.21.15.173:6815/417733706","heartbeat_front_addr":"172.21.15.173:6813/417733706","state":["exists","up"]},{"osd":10,"uuid":"46ddc07a-0a77-445e-bc19-a345e2cdce5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6817","nonce":3542783063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6819","nonce":3542783063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6823","nonce":3542783063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6821","nonce":3542783063}]},"public_addr":"172.21.15.173:6817/3542783063","cluster_addr":"172.21.15.173:6819/3542783063","heartbeat_back_addr":"172.21.15.173:6823/3542783063","heartbeat_front_addr":"172.21.15.173:6821/3542783063","state":["exists","up"]},{"osd":11,"uuid":"6476cc59-b51a-43a8-962e-1ae48119c362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6825","nonce":4131510497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6827","nonce":4131510497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6831","nonce":4131510497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6829","nonce":4131510497}]},"public_addr":"172.21.15.173:6825/4131510497","cluster_addr":"172.21.15.173:6827/4131510497","heartbeat_back_addr":"172.21.15.173:6831/4131510497","heartbeat_front_addr":"172.21.15.173:6829/4131510497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:37.337476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:41.844672+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:03.630608+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:30.349970+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:10.754534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:13.998635+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:45.526672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:51.907686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:12.388490+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:33.334404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:01.420385+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:29.765854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/2156160235":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1488046566":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6801/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6800/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/2405347332":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1860079602":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6801/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6800/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/2669724212":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/1079154737":"2024-09-14T10:26:47.425146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:33:47.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:47 smithi060 ceph-mon[27375]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-13T10:33:47.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:47 smithi060 ceph-mon[27375]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T10:33:47.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:47 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1602150712' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:47.613 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-13T10:33:47.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T10:33:47.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:47 smithi173 ceph-mon[30864]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-13T10:33:47.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:47 smithi173 ceph-mon[30864]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T10:33:47.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:47 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1602150712' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:47 smithi138 ceph-mon[31033]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-13T10:33:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:47 smithi138 ceph-mon[31033]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T10:33:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:47 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1602150712' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:47.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:48.528 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:33:48.528 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","created":"2024-09-13T10:26:24.281921+0000","modified":"2024-09-13T10:33:46.379238+0000","last_up_change":"2024-09-13T10:33:31.317557+0000","last_in_change":"2024-09-13T10:33:11.111853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T10:29:06.355752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3db36d02-fea0-4b18-8994-97158486c7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6803","nonce":756062006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6805","nonce":756062006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6809","nonce":756062006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6807","nonce":756062006}]},"public_addr":"172.21.15.60:6803/756062006","cluster_addr":"172.21.15.60:6805/756062006","heartbeat_back_addr":"172.21.15.60:6809/756062006","heartbeat_front_addr":"172.21.15.60:6807/756062006","state":["exists","up"]},{"osd":1,"uuid":"f2323c13-004c-47e1-81ed-44e6c9fb193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6811","nonce":3814817161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6813","nonce":3814817161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6817","nonce":3814817161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6815","nonce":3814817161}]},"public_addr":"172.21.15.60:6811/3814817161","cluster_addr":"172.21.15.60:6813/3814817161","heartbeat_back_addr":"172.21.15.60:6817/3814817161","heartbeat_front_addr":"172.21.15.60:6815/3814817161","state":["exists","up"]},{"osd":2,"uuid":"bda5f20f-8413-4a97-9969-1eea2f1f7b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6819","nonce":1235426811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6821","nonce":1235426811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6825","nonce":1235426811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6823","nonce":1235426811}]},"public_addr":"172.21.15.60:6819/1235426811","cluster_addr":"172.21.15.60:6821/1235426811","heartbeat_back_addr":"172.21.15.60:6825/1235426811","heartbeat_front_addr":"172.21.15.60:6823/1235426811","state":["exists","up"]},{"osd":3,"uuid":"43cba2dc-ab85-4d9a-9b2e-c4fd527077a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6827","nonce":3263875168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6829","nonce":3263875168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6833","nonce":3263875168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6831","nonce":3263875168}]},"public_addr":"172.21.15.60:6827/3263875168","cluster_addr":"172.21.15.60:6829/3263875168","heartbeat_back_addr":"172.21.15.60:6833/3263875168","heartbeat_front_addr":"172.21.15.60:6831/3263875168","state":["exists","up"]},{"osd":4,"uuid":"b76daf81-be0c-42b5-b262-a4a0cfd615f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6801","nonce":2143931671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2143931671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2143931671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2143931671}]},"public_addr":"172.21.15.138:6801/2143931671","cluster_addr":"172.21.15.138:6803/2143931671","heartbeat_back_addr":"172.21.15.138:6807/2143931671","heartbeat_front_addr":"172.21.15.138:6805/2143931671","state":["exists","up"]},{"osd":5,"uuid":"9037c1e7-d476-4980-8155-6244430c9011","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6809","nonce":2632797914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6811","nonce":2632797914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6815","nonce":2632797914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6813","nonce":2632797914}]},"public_addr":"172.21.15.138:6809/2632797914","cluster_addr":"172.21.15.138:6811/2632797914","heartbeat_back_addr":"172.21.15.138:6815/2632797914","heartbeat_front_addr":"172.21.15.138:6813/2632797914","state":["exists","up"]},{"osd":6,"uuid":"b635470b-4619-4a8a-b016-1811d5049544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6817","nonce":3026996312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6819","nonce":3026996312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6823","nonce":3026996312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6821","nonce":3026996312}]},"public_addr":"172.21.15.138:6817/3026996312","cluster_addr":"172.21.15.138:6819/3026996312","heartbeat_back_addr":"172.21.15.138:6823/3026996312","heartbeat_front_addr":"172.21.15.138:6821/3026996312","state":["exists","up"]},{"osd":7,"uuid":"4c58cdb7-9705-4a68-816f-8d6ddeca34d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6825","nonce":1381162744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6827","nonce":1381162744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6831","nonce":1381162744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6829","nonce":1381162744}]},"public_addr":"172.21.15.138:6825/1381162744","cluster_addr":"172.21.15.138:6827/1381162744","heartbeat_back_addr":"172.21.15.138:6831/1381162744","heartbeat_front_addr":"172.21.15.138:6829/1381162744","state":["exists","up"]},{"osd":8,"uuid":"b77d95ef-5b65-4446-ae22-ef2dfe094667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6801","nonce":1825049558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6803","nonce":1825049558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6807","nonce":1825049558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6805","nonce":1825049558}]},"public_addr":"172.21.15.173:6801/1825049558","cluster_addr":"172.21.15.173:6803/1825049558","heartbeat_back_addr":"172.21.15.173:6807/1825049558","heartbeat_front_addr":"172.21.15.173:6805/1825049558","state":["exists","up"]},{"osd":9,"uuid":"b89d58eb-56f0-4ede-aa1b-8d6c25be30af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6809","nonce":417733706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6811","nonce":417733706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6815","nonce":417733706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6813","nonce":417733706}]},"public_addr":"172.21.15.173:6809/417733706","cluster_addr":"172.21.15.173:6811/417733706","heartbeat_back_addr":"172.21.15.173:6815/417733706","heartbeat_front_addr":"172.21.15.173:6813/417733706","state":["exists","up"]},{"osd":10,"uuid":"46ddc07a-0a77-445e-bc19-a345e2cdce5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6817","nonce":3542783063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6819","nonce":3542783063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6823","nonce":3542783063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6821","nonce":3542783063}]},"public_addr":"172.21.15.173:6817/3542783063","cluster_addr":"172.21.15.173:6819/3542783063","heartbeat_back_addr":"172.21.15.173:6823/3542783063","heartbeat_front_addr":"172.21.15.173:6821/3542783063","state":["exists","up"]},{"osd":11,"uuid":"6476cc59-b51a-43a8-962e-1ae48119c362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6825","nonce":4131510497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6827","nonce":4131510497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6831","nonce":4131510497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6829","nonce":4131510497}]},"public_addr":"172.21.15.173:6825/4131510497","cluster_addr":"172.21.15.173:6827/4131510497","heartbeat_back_addr":"172.21.15.173:6831/4131510497","heartbeat_front_addr":"172.21.15.173:6829/4131510497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:37.337476+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:41.844672+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:03.630608+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:30.349970+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:10.754534+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:13.998635+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:45.526672+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:51.907686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:12.388490+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:32:33.334404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:01.420385+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:33:29.765854+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/2156160235":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1488046566":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6801/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6800/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/2405347332":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1860079602":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6801/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6800/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/2669724212":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/1079154737":"2024-09-14T10:26:47.425146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:33:49.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-13T10:33:49.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-13T10:33:49.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-13T10:33:49.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-13T10:33:49.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-13T10:33:49.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-13T10:33:49.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-13T10:33:49.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-13T10:33:49.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-13T10:33:49.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-13T10:33:49.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-13T10:33:49.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-13T10:33:49.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:49 smithi060 ceph-mon[27375]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:49.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:49 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4287729171' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:49.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:49 smithi173 ceph-mon[30864]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:49.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:49 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4287729171' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:49.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:49 smithi138 ceph-mon[31033]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:49.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:49 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4287729171' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:33:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.054 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.055 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.055 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.062 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.062 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:33:51.421 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:51 smithi060 ceph-mon[27375]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:51.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:51 smithi173 ceph-mon[30864]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:51.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:51 smithi138 ceph-mon[31033]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:53.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:53 smithi060 ceph-mon[27375]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:53 smithi173 ceph-mon[30864]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:53.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:53 smithi138 ceph-mon[31033]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:54.396 INFO:teuthology.orchestra.run.smithi060.stdout:330712481803 2024-09-13T10:33:54.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-13T10:33:55.652 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:55 smithi138 ceph-mon[31033]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:55.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:55 smithi060 ceph-mon[27375]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:55.707 INFO:teuthology.orchestra.run.smithi060.stdout:197568495650 2024-09-13T10:33:55.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-13T10:33:55.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:55 smithi173 ceph-mon[30864]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:56.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:56 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:33:56.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:56 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:33:56.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:56 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:33:56.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:56 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:33:56.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:56 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:33:56.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:56 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:33:57.380 INFO:teuthology.orchestra.run.smithi060.stdout:261993005080 2024-09-13T10:33:57.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-13T10:33:57.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:57 smithi060 ceph-mon[27375]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:57.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:57 smithi060 ceph-mon[27375]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T10:33:57.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:57 smithi173 ceph-mon[30864]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:57.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:57 smithi173 ceph-mon[30864]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T10:33:57.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:57 smithi138 ceph-mon[31033]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:57.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:57 smithi138 ceph-mon[31033]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T10:33:58.102 INFO:teuthology.orchestra.run.smithi060.stdout:360777252870 2024-09-13T10:33:58.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-13T10:33:58.262 INFO:teuthology.orchestra.run.smithi060.stdout:167503724583 2024-09-13T10:33:58.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-13T10:33:58.898 INFO:teuthology.orchestra.run.smithi060.stdout:90194313270 2024-09-13T10:33:58.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-13T10:33:59.375 INFO:teuthology.orchestra.run.smithi060.stdout:115964117041 2024-09-13T10:33:59.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-13T10:33:59.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:33:59 smithi060 ceph-mon[27375]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:59.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:33:59 smithi173 ceph-mon[30864]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:33:59.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:33:59 smithi138 ceph-mon[31033]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:01.365 INFO:teuthology.orchestra.run.smithi060.stdout:223338299423 2024-09-13T10:34:01.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-13T10:34:01.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:01 smithi060 ceph-mon[27375]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:01 smithi173 ceph-mon[30864]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:01.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:01 smithi138 ceph-mon[31033]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:01.912 INFO:teuthology.orchestra.run.smithi060.stdout:296352743444 2024-09-13T10:34:01.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-13T10:34:02.814 INFO:teuthology.orchestra.run.smithi060.stdout:141733920815 2024-09-13T10:34:02.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-13T10:34:03.139 INFO:teuthology.orchestra.run.smithi060.stdout:34359738433 2024-09-13T10:34:03.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-13T10:34:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:03 smithi173 ceph-mon[30864]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:03.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:03 smithi060 ceph-mon[27375]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:03.820 INFO:teuthology.orchestra.run.smithi060.stdout:55834574912 2024-09-13T10:34:03.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-13T10:34:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:03 smithi138 ceph-mon[31033]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:03.905 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:03.923 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:04.161 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:04.162 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:04.841 INFO:teuthology.orchestra.run.smithi060.stdout:197568495652 2024-09-13T10:34:05.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:05 smithi060 ceph-mon[27375]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:05.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:05 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/672931081' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T10:34:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:05 smithi173 ceph-mon[30864]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:05 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/672931081' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T10:34:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:05 smithi138 ceph-mon[31033]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:05 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/672931081' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T10:34:06.559 INFO:teuthology.orchestra.run.smithi060.stdout:90194313272 2024-09-13T10:34:06.732 INFO:teuthology.orchestra.run.smithi060.stdout:296352743445 2024-09-13T10:34:07.147 INFO:teuthology.orchestra.run.smithi060.stdout:261993005082 2024-09-13T10:34:07.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:07 smithi060 ceph-mon[27375]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:07.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:07 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3311410196' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T10:34:07.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:07 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/171545066' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-13T10:34:07.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:07 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1400979770' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-13T10:34:07.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743444 got 296352743445 for osd.9 2024-09-13T10:34:07.776 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:07.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:07 smithi173 ceph-mon[30864]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:07.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:07 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3311410196' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T10:34:07.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:07 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/171545066' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-13T10:34:07.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:07 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1400979770' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-13T10:34:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:07 smithi138 ceph-mon[31033]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:07 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3311410196' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T10:34:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:07 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/171545066' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-13T10:34:07.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:07 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1400979770' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-13T10:34:08.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495650 got 197568495652 for osd.6 2024-09-13T10:34:08.010 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:08.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:08 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/167637884' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-13T10:34:08.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:08 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1567444398' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T10:34:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:08 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/167637884' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-13T10:34:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:08 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1567444398' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T10:34:08.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:08 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/167637884' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-13T10:34:08.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:08 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1567444398' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T10:34:09.024 INFO:teuthology.orchestra.run.smithi060.stdout:141733920816 2024-09-13T10:34:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:09 smithi060 ceph-mon[27375]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:09.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:09 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2094304249' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T10:34:09.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:09 smithi173 ceph-mon[30864]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:09.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:09 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2094304249' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T10:34:09.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920815 got 141733920816 for osd.4 2024-09-13T10:34:09.794 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:09 smithi138 ceph-mon[31033]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:09 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2094304249' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T10:34:09.908 INFO:teuthology.orchestra.run.smithi060.stdout:330712481806 2024-09-13T10:34:10.389 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481803 got 330712481806 for osd.10 2024-09-13T10:34:10.390 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:10.688 INFO:teuthology.orchestra.run.smithi060.stdout:115964117044 2024-09-13T10:34:10.791 INFO:teuthology.orchestra.run.smithi060.stdout:167503724586 2024-09-13T10:34:11.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:11 smithi060 ceph-mon[27375]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:11.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:11 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2459513014' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T10:34:11.515 INFO:teuthology.orchestra.run.smithi060.stdout:223338299425 2024-09-13T10:34:11.754 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117041 got 115964117044 for osd.3 2024-09-13T10:34:11.755 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:11.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:11 smithi173 ceph-mon[30864]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:11.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:11 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2459513014' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T10:34:11.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:11 smithi138 ceph-mon[31033]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:11.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:11 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2459513014' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T10:34:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:12.387 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313270 got 90194313272 for osd.2 2024-09-13T10:34:12.387 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:12 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:12 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:12 smithi060 ceph-mon[27375]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T10:34:12.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:12 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2155368185' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T10:34:12.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:12 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:12.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:12 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:12.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:12 smithi173 ceph-mon[30864]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T10:34:12.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:12 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2155368185' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T10:34:12.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005080 got 261993005082 for osd.8 2024-09-13T10:34:12.808 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:12.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:12 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:12.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:12 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:12.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:12 smithi138 ceph-mon[31033]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T10:34:12.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:12 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2155368185' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T10:34:13.254 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299423 got 223338299425 for osd.7 2024-09-13T10:34:13.255 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:13 smithi173 ceph-mon[30864]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:13 smithi138 ceph-mon[31033]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:13.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:13 smithi060 ceph-mon[27375]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:13.889 INFO:teuthology.orchestra.run.smithi060.stdout:360777252874 2024-09-13T10:34:14.165 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:14.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724583 got 167503724586 for osd.5 2024-09-13T10:34:14.200 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:14.617 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:14.721 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:14 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:34:14.721 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:14 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:34:14.722 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:14 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3847215739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-13T10:34:14.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:14 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:34:14.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:14 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:34:14.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:14 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3847215739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-13T10:34:14.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:14 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:34:14.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:14 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:34:14.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:14 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3847215739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-13T10:34:15.016 INFO:teuthology.orchestra.run.smithi060.stdout:55834574914 2024-09-13T10:34:15.396 INFO:tasks.cephadm.ceph_manager.ceph:need seq 360777252870 got 360777252874 for osd.11 2024-09-13T10:34:15.396 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:15.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:15 smithi060 ceph-mon[27375]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:15.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:15 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3898313111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T10:34:15.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:15 smithi173 ceph-mon[30864]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:15.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:15 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3898313111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T10:34:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:15 smithi138 ceph-mon[31033]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:15 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3898313111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T10:34:16.407 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574914 for osd.1 2024-09-13T10:34:16.407 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:16.787 INFO:teuthology.orchestra.run.smithi060.stdout:34359738439 2024-09-13T10:34:17.554 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738433 got 34359738439 for osd.0 2024-09-13T10:34:17.555 DEBUG:teuthology.parallel:result is None 2024-09-13T10:34:17.555 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-13T10:34:17.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:34:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:17 smithi173 ceph-mon[30864]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:17 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:17 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:17 smithi173 ceph-mon[30864]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T10:34:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:17 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/853346841' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T10:34:17.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:17 smithi138 ceph-mon[31033]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:17 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:17 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:17 smithi138 ceph-mon[31033]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T10:34:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:17 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/853346841' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T10:34:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:17 smithi060 ceph-mon[27375]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:17 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:17 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:17 smithi060 ceph-mon[27375]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T10:34:17.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:17 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/853346841' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T10:34:18.449 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:34:19.092 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-13T10:34:17.974443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":337624,"kb_used_data":9128,"kb_used_omap":18,"kb_used_meta":328237,"kb_avail":1124604200,"statfs":{"total":1151940427776,"available":1151594700800,"internally_reserved":0,"allocated":9347072,"data_stored":5337430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":336115002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001557"},"pg_stats":[{"pgid":"1.0","version":"16'92","reported_seq":133,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T10:34:13.003713+0000","last_change":"2024-09-13T10:34:13.003713+0000","last_active":"2024-09-13T10:34:13.003713+0000","last_peered":"2024-09-13T10:34:13.003713+0000","last_clean":"2024-09-13T10:34:13.003713+0000","last_became_active":"2024-09-13T10:33:33.765804+0000","last_became_peered":"2024-09-13T10:33:33.765804+0000","last_unstale":"2024-09-13T10:34:13.003713+0000","last_undegraded":"2024-09-13T10:34:13.003713+0000","last_fullsized":"2024-09-13T10:34:13.003713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:16.331942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3727360,"data_stored":3702496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1220608,"data_stored":884313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481808,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1220608,"data_stored":884333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117046,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713292,"statfs":{"total":95995035648,"available":95962411008,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:34:19.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:34:19.347 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:19.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:19 smithi060 ceph-mon[27375]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:19.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:19 smithi060 ceph-mon[27375]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:19.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:19 smithi173 ceph-mon[30864]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:19.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:19 smithi173 ceph-mon[30864]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:19.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:19 smithi138 ceph-mon[31033]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:19.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:19 smithi138 ceph-mon[31033]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:20.006 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:20.006 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:34:20.751 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-13T10:34:19.974911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":337660,"kb_used_data":9164,"kb_used_omap":18,"kb_used_meta":328237,"kb_avail":1124604164,"statfs":{"total":1151940427776,"available":1151594663936,"internally_reserved":0,"allocated":9383936,"data_stored":5359561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":336115002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.519691"},"pg_stats":[{"pgid":"1.0","version":"16'92","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T10:34:17.482729+0000","last_change":"2024-09-13T10:34:13.003713+0000","last_active":"2024-09-13T10:34:17.482729+0000","last_peered":"2024-09-13T10:34:17.482729+0000","last_clean":"2024-09-13T10:34:17.482729+0000","last_became_active":"2024-09-13T10:33:33.765804+0000","last_became_peered":"2024-09-13T10:33:33.765804+0000","last_unstale":"2024-09-13T10:34:17.482729+0000","last_undegraded":"2024-09-13T10:34:17.482729+0000","last_fullsized":"2024-09-13T10:34:17.482729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:16.331942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3727360,"data_stored":3702496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1220608,"data_stored":884313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481809,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1232896,"data_stored":891710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27700,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713292,"statfs":{"total":95995035648,"available":95962411008,"internally_reserved":0,"allocated":688128,"data_stored":355405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:34:20.752 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-13T10:34:20.752 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T10:34:20.752 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-13T10:34:20.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-13T10:34:21.010 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:21.697 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:21.698 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T10:34:21.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:21 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:21.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:21 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:21.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:21 smithi060 ceph-mon[27375]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:21.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:21 smithi060 ceph-mon[27375]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:21.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:21 smithi060 ceph-mon[27375]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T10:34:21.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:21 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:21.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:21 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:21.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:21 smithi173 ceph-mon[30864]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:21.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:21 smithi173 ceph-mon[30864]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:21.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:21 smithi173 ceph-mon[30864]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T10:34:21.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:21 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:21.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:21 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:21.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:21 smithi138 ceph-mon[31033]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:21.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:21 smithi138 ceph-mon[31033]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:21.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:21 smithi138 ceph-mon[31033]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T10:34:22.400 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-13T10:34:22.400 INFO:tasks.cephadm:Setup complete, yielding 2024-09-13T10:34:22.400 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T10:34:22.412 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T10:34:22.412 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-aclamk-cbt-combined'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'filestore odsync write': True, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', '\\(MON_DOWN\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-aclamk-cbt-combined', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}} 2024-09-13T10:34:22.412 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T10:34:22.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T10:34:22.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:22 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/769012130' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-13T10:34:22.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:22 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:22.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:22 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:22.674 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:22.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:22 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/769012130' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-13T10:34:22.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:22 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:22.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:22 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:22.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:22 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/769012130' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-13T10:34:22.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:22 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:22.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:22 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:23.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:23 smithi060 ceph-mon[27375]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:23.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:23 smithi060 ceph-mon[27375]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T10:34:23.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:23 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1829454219' entity='client.admin' 2024-09-13T10:34:23.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:23 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:34:23.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:23 smithi173 ceph-mon[30864]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:23.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:23 smithi173 ceph-mon[30864]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T10:34:23.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:23 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1829454219' entity='client.admin' 2024-09-13T10:34:23.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:23 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:34:23.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:23 smithi138 ceph-mon[31033]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:23.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:23 smithi138 ceph-mon[31033]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T10:34:23.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:23 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1829454219' entity='client.admin' 2024-09-13T10:34:23.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:23 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:34:24.008 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T10:34:24.008 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-13T10:34:24.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:34:24.267 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:24.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:24 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:24.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:24 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:24.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:34:24.750 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:34:24.750 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:24 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:34:24.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:24 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:24.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:24 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:24.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:34:24.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:34:24.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:24 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:34:24.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:24 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:24.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:24 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:24.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:34:24.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:34:24.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:24 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:34:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:34:25.609 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-13T10:34:24.505702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":337844,"kb_used_data":9344,"kb_used_omap":18,"kb_used_meta":328237,"kb_avail":1124603980,"statfs":{"total":1151940427776,"available":1151594475520,"internally_reserved":0,"allocated":9568256,"data_stored":5470216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":336115002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.531259"},"pg_stats":[{"pgid":"1.0","version":"16'92","reported_seq":139,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T10:34:22.941669+0000","last_change":"2024-09-13T10:34:13.003713+0000","last_active":"2024-09-13T10:34:22.941669+0000","last_peered":"2024-09-13T10:34:22.941669+0000","last_clean":"2024-09-13T10:34:22.941669+0000","last_became_active":"2024-09-13T10:33:33.765804+0000","last_became_peered":"2024-09-13T10:33:33.765804+0000","last_unstale":"2024-09-13T10:34:22.941669+0000","last_undegraded":"2024-09-13T10:34:22.941669+0000","last_fullsized":"2024-09-13T10:34:22.941669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:16.331942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3727360,"data_stored":3702496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1232896,"data_stored":891690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743449,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005086,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28260,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716892,"statfs":{"total":95995035648,"available":95966097408,"internally_reserved":0,"allocated":1257472,"data_stored":906464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":724992,"data_stored":377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713280,"statfs":{"total":95995035648,"available":95962398720,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:34:25.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:34:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:25 smithi138 ceph-mon[31033]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:25.769 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:25 smithi138 ceph-mon[31033]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T10:34:25.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:25 smithi173 ceph-mon[30864]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:25.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:25 smithi173 ceph-mon[30864]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T10:34:25.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:25.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:25 smithi060 ceph-mon[27375]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:25.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:25 smithi060 ceph-mon[27375]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T10:34:26.477 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:26.477 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:34:26.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:26 smithi060 ceph-mon[27375]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:26.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:26 smithi173 ceph-mon[30864]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:26.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:26 smithi138 ceph-mon[31033]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-13T10:34:25.976091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":2,"kb":1124941824,"kb_used":337932,"kb_used_data":9428,"kb_used_omap":18,"kb_used_meta":328237,"kb_avail":1124603892,"statfs":{"total":1151940427776,"available":1151594385408,"internally_reserved":0,"allocated":9654272,"data_stored":5521855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":336115002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001179"},"pg_stats":[{"pgid":"1.0","version":"16'92","reported_seq":139,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T10:34:22.941669+0000","last_change":"2024-09-13T10:34:13.003713+0000","last_active":"2024-09-13T10:34:22.941669+0000","last_peered":"2024-09-13T10:34:22.941669+0000","last_clean":"2024-09-13T10:34:22.941669+0000","last_became_active":"2024-09-13T10:33:33.765804+0000","last_became_peered":"2024-09-13T10:33:33.765804+0000","last_unstale":"2024-09-13T10:34:22.941669+0000","last_undegraded":"2024-09-13T10:34:22.941669+0000","last_fullsized":"2024-09-13T10:34:22.941669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:16.331942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3727360,"data_stored":3702496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1232896,"data_stored":891690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743449,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005086,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28260,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716892,"statfs":{"total":95995035648,"available":95966097408,"internally_reserved":0,"allocated":1257472,"data_stored":906464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717404,"statfs":{"total":95995035648,"available":95966621696,"internally_reserved":0,"allocated":737280,"data_stored":384913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":724992,"data_stored":377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117048,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27740,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717412,"statfs":{"total":95995035648,"available":95966629888,"internally_reserved":0,"allocated":724992,"data_stored":377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313276,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717440,"statfs":{"total":95995035648,"available":95966658560,"internally_reserved":0,"allocated":700416,"data_stored":362782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31896,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":724992,"data_stored":377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":712704,"data_stored":370159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:34:27.175 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-13T10:34:27.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump 2024-09-13T10:34:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:27 smithi173 ceph-mon[30864]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:27.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:27 smithi173 ceph-mon[30864]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:27.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:27 smithi060 ceph-mon[27375]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:27.800 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:27 smithi060 ceph-mon[27375]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:27 smithi138 ceph-mon[31033]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:27.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:27 smithi138 ceph-mon[31033]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:28.083 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 95 2024-09-13T10:34:28.083 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-09-13T10:26:24.281921+0000 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-09-13T10:34:24.500562+0000 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:crush_version 26 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:full_ratio 0.95 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:backfillfull_ratio 0.9 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:nearfull_ratio 0.85 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:require_min_compat_client luminous 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:min_compat_client jewel 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:require_osd_release squid 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:stretch_mode_enabled false 2024-09-13T10:34:28.084 INFO:teuthology.orchestra.run.smithi060.stdout:pool 1 '.mgr' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 11.76 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:max_osd 12 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 up in weight 1 up_from 8 up_thru 52 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006] [v2:172.21.15.60:6804/756062006,v1:172.21.15.60:6805/756062006] exists,up 3db36d02-fea0-4b18-8994-97158486c7cd 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161] [v2:172.21.15.60:6812/3814817161,v1:172.21.15.60:6813/3814817161] exists,up f2323c13-004c-47e1-81ed-44e6c9fb193a 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811] [v2:172.21.15.60:6820/1235426811,v1:172.21.15.60:6821/1235426811] exists,up bda5f20f-8413-4a97-9969-1eea2f1f7b87 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168] [v2:172.21.15.60:6828/3263875168,v1:172.21.15.60:6829/3263875168] exists,up 43cba2dc-ab85-4d9a-9b2e-c4fd527077a0 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671] [v2:172.21.15.138:6802/2143931671,v1:172.21.15.138:6803/2143931671] exists,up b76daf81-be0c-42b5-b262-a4a0cfd615f8 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 up in weight 1 up_from 39 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914] [v2:172.21.15.138:6810/2632797914,v1:172.21.15.138:6811/2632797914] exists,up 9037c1e7-d476-4980-8155-6244430c9011 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 up in weight 1 up_from 46 up_thru 47 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312] [v2:172.21.15.138:6818/3026996312,v1:172.21.15.138:6819/3026996312] exists,up b635470b-4619-4a8a-b016-1811d5049544 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 up in weight 1 up_from 52 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744] [v2:172.21.15.138:6826/1381162744,v1:172.21.15.138:6827/1381162744] exists,up 4c58cdb7-9705-4a68-816f-8d6ddeca34d3 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8 up in weight 1 up_from 61 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558] [v2:172.21.15.173:6802/1825049558,v1:172.21.15.173:6803/1825049558] exists,up b77d95ef-5b65-4446-ae22-ef2dfe094667 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9 up in weight 1 up_from 69 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706] [v2:172.21.15.173:6810/417733706,v1:172.21.15.173:6811/417733706] exists,up b89d58eb-56f0-4ede-aa1b-8d6c25be30af 2024-09-13T10:34:28.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10 up in weight 1 up_from 77 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063] [v2:172.21.15.173:6818/3542783063,v1:172.21.15.173:6819/3542783063] exists,up 46ddc07a-0a77-445e-bc19-a345e2cdce5d 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11 up in weight 1 up_from 84 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497] [v2:172.21.15.173:6826/4131510497,v1:172.21.15.173:6827/4131510497] exists,up 6476cc59-b51a-43a8-962e-1ae48119c362 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2156160235 expires 2024-09-14T10:27:11.894382+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1488046566 expires 2024-09-14T10:27:11.894382+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/250706696 expires 2024-09-14T10:27:11.894382+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/250706696 expires 2024-09-14T10:27:11.894382+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2405347332 expires 2024-09-14T10:27:11.894382+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1860079602 expires 2024-09-14T10:26:47.425146+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/3783615351 expires 2024-09-14T10:26:47.425146+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/3783615351 expires 2024-09-14T10:26:47.425146+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2669724212 expires 2024-09-14T10:26:47.425146+0000 2024-09-13T10:34:28.086 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1079154737 expires 2024-09-14T10:26:47.425146+0000 2024-09-13T10:34:28.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:28 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3931154940' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-13T10:34:28.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:28 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3931154940' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-13T10:34:28.871 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:28 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3931154940' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-13T10:34:28.907 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 52 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/756062006,v1:172.21.15.60:6803/756062006] [v2:172.21.15.60:6804/756062006,v1:172.21.15.60:6805/756062006] exists,up 3db36d02-fea0-4b18-8994-97158486c7cd', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/3814817161,v1:172.21.15.60:6811/3814817161] [v2:172.21.15.60:6812/3814817161,v1:172.21.15.60:6813/3814817161] exists,up f2323c13-004c-47e1-81ed-44e6c9fb193a', 'osd.2 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/1235426811,v1:172.21.15.60:6819/1235426811] [v2:172.21.15.60:6820/1235426811,v1:172.21.15.60:6821/1235426811] exists,up bda5f20f-8413-4a97-9969-1eea2f1f7b87', 'osd.3 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/3263875168,v1:172.21.15.60:6827/3263875168] [v2:172.21.15.60:6828/3263875168,v1:172.21.15.60:6829/3263875168] exists,up 43cba2dc-ab85-4d9a-9b2e-c4fd527077a0', 'osd.4 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/2143931671,v1:172.21.15.138:6801/2143931671] [v2:172.21.15.138:6802/2143931671,v1:172.21.15.138:6803/2143931671] exists,up b76daf81-be0c-42b5-b262-a4a0cfd615f8', 'osd.5 up in weight 1 up_from 39 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/2632797914,v1:172.21.15.138:6809/2632797914] [v2:172.21.15.138:6810/2632797914,v1:172.21.15.138:6811/2632797914] exists,up 9037c1e7-d476-4980-8155-6244430c9011', 'osd.6 up in weight 1 up_from 46 up_thru 47 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/3026996312,v1:172.21.15.138:6817/3026996312] [v2:172.21.15.138:6818/3026996312,v1:172.21.15.138:6819/3026996312] exists,up b635470b-4619-4a8a-b016-1811d5049544', 'osd.7 up in weight 1 up_from 52 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/1381162744,v1:172.21.15.138:6825/1381162744] [v2:172.21.15.138:6826/1381162744,v1:172.21.15.138:6827/1381162744] exists,up 4c58cdb7-9705-4a68-816f-8d6ddeca34d3', 'osd.8 up in weight 1 up_from 61 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/1825049558,v1:172.21.15.173:6801/1825049558] [v2:172.21.15.173:6802/1825049558,v1:172.21.15.173:6803/1825049558] exists,up b77d95ef-5b65-4446-ae22-ef2dfe094667', 'osd.9 up in weight 1 up_from 69 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/417733706,v1:172.21.15.173:6809/417733706] [v2:172.21.15.173:6810/417733706,v1:172.21.15.173:6811/417733706] exists,up b89d58eb-56f0-4ede-aa1b-8d6c25be30af', 'osd.10 up in weight 1 up_from 77 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/3542783063,v1:172.21.15.173:6817/3542783063] [v2:172.21.15.173:6818/3542783063,v1:172.21.15.173:6819/3542783063] exists,up 46ddc07a-0a77-445e-bc19-a345e2cdce5d', 'osd.11 up in weight 1 up_from 84 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/4131510497,v1:172.21.15.173:6825/4131510497] [v2:172.21.15.173:6826/4131510497,v1:172.21.15.173:6827/4131510497] exists,up 6476cc59-b51a-43a8-962e-1ae48119c362'] 2024-09-13T10:34:28.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon mon.a config show 2024-09-13T10:34:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:29.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:29 smithi060 ceph-mon[27375]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:29.678 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mon.a", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster": "ceph", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "admin_socket_mode": "", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_client_required": "cephx, none", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_cluster_required": "cephx", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_debug": "true", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T10:34:29.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_service_required": "cephx", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_supported": "", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio": "true", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio_reap_max": "16", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_async_discard_threads": "0", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_block_size": "4096", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_aio": "false", 2024-09-13T10:34:29.680 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_enable_discard": "false", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_flock_retry": "3", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_inject_crash": "0", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_ioring": "false", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_ioring_hipri": "false", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T10:34:29.681 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_type": "", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_allocator": "hybrid", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_buffered_io": "true", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T10:34:29.682 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_replay_recovery": "false", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T10:34:29.683 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_sync_write": "false", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_allocator": "hybrid", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_async_db_compaction": "true", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bdev_label_multi": "true", 2024-09-13T10:34:29.684 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_create": "true", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_db_create": "false", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_db_path": "", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_db_size": "0", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_path": "", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T10:34:29.685 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_size": "107374182400", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_wal_create": "false", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_wal_path": "", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs": "true", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T10:34:29.686 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_autotune": "true", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_size": "0", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_type": "2q", 2024-09-13T10:34:29.687 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_clone_cow": "true", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_mode": "none", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T10:34:29.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_freelist": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T10:34:29.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T10:34:29.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fail_eio": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T10:34:29.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T10:34:29.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_blob_size": "0", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T10:34:29.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_mem": "512", 2024-09-13T10:34:29.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_tracing": "false", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T10:34:29.695 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_write_v2": "false", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_write_v2_random": "false", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T10:34:29.696 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_cluster_require_version": "2", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_require_signatures": "false", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_require_version": "2", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_service_require_signatures": "false", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_service_require_version": "2", 2024-09-13T10:34:29.697 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_sign_messages": "true", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "chdir": "", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_acl_type": "", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_asio_thread_count": "2", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_cache_mid": "0.750000", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_cache_size": "16384", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_caps_release_delay": "5", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_check_pool_perm": "true", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_force_sync_read": "false", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_getattr_caps": "false", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_inject_features": "", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T10:34:29.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_die_on_failed_remount": "false", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_dirsize_rbytes": "true", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_force_lazyio": "false", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_fs": "", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_inject_release_failure": "false", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_max_inline_size": "4096", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mds_namespace": "", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_metadata": "", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mount_gid": "-1", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mount_timeout": "300", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mount_uid": "-1", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mountpoint": "/", 2024-09-13T10:34:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_notify_timeout": "10", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc": "true", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_max_objects": "1000", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_size": "209715200", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_permissions": "true", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_quota": "true", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_quota_df": "true", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_readahead_max_bytes": "0", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_readahead_max_periods": "4", 2024-09-13T10:34:29.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_readahead_min": "131072", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_reconnect_stale": "false", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_shutdown_timeout": "30", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_snapdir": ".snap", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_tick_interval": "1", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_trace": "", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_use_faked_inos": "false", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_use_random_mds": "false", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_graylog": "false", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_graylog_port": "12201", 2024-09-13T10:34:29.701 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_monitors": "default=true", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_syslog": "false", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_syslog_level": "info", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "-", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_network": "", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_network_interface": "", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zlib_isal": "false", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zlib_level": "5", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zstd_level": "1", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f", 2024-09-13T10:34:29.702 INFO:teuthology.orchestra.run.smithi060.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_osd_stat_interval": "0", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_seastar_cpu_cores": "", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_seastar_num_threads": "0", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_location": "", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_location_hook": "", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_location_hook_timeout": "10", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "daemonize": "false", 2024-09-13T10:34:29.703 INFO:teuthology.orchestra.run.smithi060.stdout: "daos_pool": "tank", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_alienstore": "0/5", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_asok": "1/5", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_asok_assert_abort": "false", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_auth": "1/5", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_bdev": "1/3", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_bluefs": "1/5", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_bluestore": "1/5", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_buffer": "0/1", 2024-09-13T10:34:29.704 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_ceph_dedup": "0/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_cephsqlite": "0/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_client": "0/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_compressor": "1/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_context": "0/1", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_crimson_interrupt": "0/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_crush": "1/1", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_crypto": "1/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_cyanstore": "0/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_dpdk": "1/5", 2024-09-13T10:34:29.705 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_eventtrace": "1/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_filer": "0/1", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_filestore": "1/3", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_finisher": "1/1", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_fuse": "1/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_javaclient": "1/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_journal": "1/3", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_journaler": "0/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_kstore": "1/5", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_lockdep": "0/1", 2024-09-13T10:34:29.706 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mclock": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_balancer": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_locker": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_log": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_migrator": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_quiesce": "3/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_memstore": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mgr": "2/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mgrc": "1/5", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mon": "20/20", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_monc": "0/10", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_ms": "1/1", 2024-09-13T10:34:29.707 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_none": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_objclass": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_objectcacher": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_objecter": "0/1", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_optracker": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_osd": "1/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_paxos": "20/20", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_perfcounter": "1/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_prioritycache": "1/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rados": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd_replay": "0/5", 2024-09-13T10:34:29.708 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_refs": "0/0", 2024-09-13T10:34:29.709 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_reserver": "1/1", 2024-09-13T10:34:29.709 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw": "1/5", 2024-09-13T10:34:29.709 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_access": "1/5", 2024-09-13T10:34:29.709 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T10:34:29.709 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T10:34:29.709 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_flight": "1/5", 2024-09-13T10:34:29.710 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-13T10:34:29.710 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_notification": "1/5", 2024-09-13T10:34:29.710 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_sync": "1/5", 2024-09-13T10:34:29.710 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rocksdb": "4/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_backref": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_cache": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_device": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_epm": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_journal": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_lba": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_odata": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_omap": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_onode": "0/5", 2024-09-13T10:34:29.711 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_t": "0/5", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_tm": "0/5", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_striper": "0/1", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_test": "0/5", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_throttle": "1/1", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_timer": "0/1", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_tp": "0/5", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_trace": "1/5", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "device_failure_prediction_mode": "none", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_graylog": "false", 2024-09-13T10:34:29.712 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_journald": "false", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_stderr": "true", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_syslog": "false", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event_tracing": "false", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_cert_file": "", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_http_port": "9926", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_key_file": "", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_prio_limit": "5", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_sort_metrics": "true", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_stats_period": "5", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "fake_statfs_for_testing": "0", 2024-09-13T10:34:29.713 INFO:teuthology.orchestra.run.smithi060.stdout: "fatal_signal_handlers": "true", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filer_max_purge_ops": "10", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filer_max_truncate_ops": "128", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_blackhole": "false", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_btrfs_snap": "true", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_caller_concurrency": "10", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_omap_check": "false", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_verify_split": "false", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_dump_file": "", 2024-09-13T10:34:29.714 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fadvise": "true", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fail_eio": "true", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fd_cache_size": "128", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fiemap": "false", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_inject_stall": "0", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_journal_parallel": "false", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_journal_trailing": "false", 2024-09-13T10:34:29.715 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_journal_writeahead": "false", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_kill_at": "0", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T10:34:29.716 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_merge_threshold": "-10", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_odsync_write": "false", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_omap_backend_path": "", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_op_threads": "2", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_punch_hole": "false", 2024-09-13T10:34:29.717 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_ops": "50", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_seek_data_hole": "false", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_sloppy_crc": "false", 2024-09-13T10:34:29.718 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_splice": "false", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_split_multiple": "2", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_split_rand_factor": "20", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_update_to": "1000", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T10:34:29.719 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_xfs_extsize": "false", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_zfs_snap": "false", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "fio_dir": "/tmp/fio", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "74376cb6-71ba-11ef-bceb-c7b262605968", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_allow_other": "true", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_big_writes": "true", 2024-09-13T10:34:29.720 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_debug": "false", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_default_permissions": "false", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_disable_pagecache": "false", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_max_write": "0", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_multithreaded": "true", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_require_active_mds": "true", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_set_user_groups": "true", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_splice_move": "true", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_splice_read": "true", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_splice_write": "true", 2024-09-13T10:34:29.721 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "gss_target_name": "ceph", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_file": "", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_inject_failure": "0", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_interval": "5", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "host": "smithi060", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T10:34:29.722 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "inject_early_sigterm": "false", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "jaeger_agent_port": "6799", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "jaeger_tracing_enable": "false", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_aio": "true", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_align_min_size": "65536", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_block_align": "true", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_block_size": "4096", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_dio": "true", 2024-09-13T10:34:29.723 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_discard": "false", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_force_aio": "false", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_ignore_corruption": "false", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_max_write_entries": "100", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_replay_from": "0", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_write_header_frequency": "0", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_zero_on_create": "false", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journaler_prefetch_periods": "10", 2024-09-13T10:34:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: "journaler_prezero_periods": "5", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "journaler_write_head_interval": "15", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "keyfile": "", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_backend": "rocksdb", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_max_bytes": "67108864", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_max_ops": "512", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_onode_map_size": "1024", 2024-09-13T10:34:29.725 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_sync_transaction": "false", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "librados_thread_count": "2", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "lockdep": "false", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "lockdep_force_backtrace": "false", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_coarse_timestamps": "true", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_flush_on_exit": "false", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_graylog_port": "12201", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_max_new": "1000", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_max_recent": "10000", 2024-09-13T10:34:29.726 INFO:teuthology.orchestra.run.smithi060.stdout: "log_stderr_prefix": "", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": "true", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_graylog": "false", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_journald": "false", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_stderr": "false", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_syslog": "false", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_action_on_write_error": "1", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_alternate_name_max": "8192", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_asio_thread_count": "2", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_export_pin": "true", 2024-09-13T10:34:29.727 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_interval": "10", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_max": "-1", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_max_until": "-1", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_merge_size": "50", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T10:34:29.728 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_mode": "0", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_overload_epochs": "2", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_bits": "3", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_size": "10000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T10:34:29.729 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_mid": "0.700000", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_delay": "0", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T10:34:29.730 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_trim_interval": "1", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_auth_pins": "false", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_frag": "false", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_scatterstat": "false", 2024-09-13T10:34:29.731 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_subtrees": "false", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_default_dir_hash": "2", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_defer_session_stale": "true", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_max_entries": "0", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_prefetch": "true", 2024-09-13T10:34:29.732 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_early_reply": "true", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_enable_op_tracker": "true", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_enforce_unique_name": "true", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T10:34:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_health_dummy": "false", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T10:34:29.734 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_join_fs": "", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_journal_format": "1", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_create_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_dirfrag_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_export_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_import_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_journal_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_link_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_openc_at": "0", 2024-09-13T10:34:29.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_rename_at": "0", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_shutdown_at": "0", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_event_large_threshold": "524288", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_max_events": "-1", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_max_segments": "128", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_pause": "false", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_segment_size": "0", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_trim_threshold": "128", 2024-09-13T10:34:29.736 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_completed_requests": "100000", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_export_size": "20971520", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_file_recover": "32", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_purge_files": "64", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_purge_ops": "8192", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_metrics_update_interval": "2", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_min_caps_per_client": "100", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T10:34:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_numa_node": "-1", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_duration": "600", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_size": "20", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_log_threshold": "5", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_ping_grace": "15", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_ping_interval": "5", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T10:34:29.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_max_caps": "30000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_replay_interval": "1.000000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_root_ino_gid": "0", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_root_ino_uid": "0", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-13T10:34:29.739 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_shutdown_check": "0", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_skip_ino": "0", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_snap_min_uid": "0", 2024-09-13T10:34:29.740 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_snap_rstat": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_symlink_recovery": "true", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_thrash_exports": "0", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_thrash_fragments": "0", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_tick_interval": "5.000000", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_valgrind_exit": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_verify_backtrace": "1", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_verify_scatter": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_wipe_sessions": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool_debug": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T10:34:29.741 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_page_set": "false", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_page_size": "65536", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_client_bytes": "134217728", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_client_messages": "512", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_disabled_modules": "", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_enable_op_tracker": "true", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_max_pg_creating": "1024", 2024-09-13T10:34:29.742 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mds_messages": "128", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mon_messages": "128", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_duration": "600", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_size": "20", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_log_threshold": "5", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T10:34:29.743 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_osd_messages": "8192", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_pool": "true", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_standby_modules": "true", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_stats_period": "5", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_stats_threshold": "5", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_tick_period": "2", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_allow_pool_delete": "true", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_allow_pool_size_one": "false", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T10:34:29.744 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_bytes": "104857600", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_target_rank": "-1", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T10:34:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_level": "debug", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_journald": "true", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_compact_on_start": "false", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_compact_on_trim": "true", 2024-09-13T10:34:29.746 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cpu_threads": "4", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data_avail_crit": "5", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data_avail_warn": "5", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_dump_json": "false", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T10:34:29.747 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_extra_checks": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_require_squid": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_down_mkfs_grace": "60", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_down_uptime_grace": "60", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_election_default_strategy": "1", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_election_timeout": "5.000000", 2024-09-13T10:34:29.748 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_enable_op_tracker": "true", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_fake_pool_delete": "false", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_force_quorum_join": "false", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_detail_to_clog": "true", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_log_update_period": "5", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_max_detail": "50", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_to_clog": "true", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T10:34:29.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_host": "[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_host_override": "", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_initial_members": "", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_lease": "5.000000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_log_full_interval": "50", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_log_max": "10000", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_log_max_summary": "50", 2024-09-13T10:34:29.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_log_epochs": "500", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_osd": "10000", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_memory_autotune": "true", 2024-09-13T10:34:29.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_memory_target": "2147483648", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_digest_period": "5", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_min_osdmap_epochs": "50", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_complaint_time": "30", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_duration": "600", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_size": "20", 2024-09-13T10:34:29.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_log_threshold": "5", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_cache_size": "500", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T10:34:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T10:34:29.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_report_timeout": "900", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-13T10:34:29.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_max_osds": "4", 2024-09-13T10:34:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_interval": "300", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_max_keys": "100", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_timeout": "300", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_session_timeout": "300", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_smart_report_timeout": "5", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T10:34:29.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_pool_size": "4", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_debug": "false", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_tick_interval": "5", 2024-09-13T10:34:29.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T10:34:29.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_older_version": "true", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "monmap": "", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T10:34:29.760 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_tracing_enabled": "false", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_op_threads": "3", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_cm": "false", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_device_name": "", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T10:34:29.761 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_sl": "3", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_type": "ib", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_reap_threshold": "5", 2024-09-13T10:34:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_before_connect": "false", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_ipv4": "true", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_ipv6": "false", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_msgr1": "true", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_msgr2": "true", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_port_max": "7568", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_port_min": "6800", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_retry_count": "3", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_retry_delay": "5", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_client": "false", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_mds": "false", 2024-09-13T10:34:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_mgr": "false", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_mon": "false", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_osd": "false", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_client_mode": "crc secure", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_cluster_type": "", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_compress_secure": "false", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_crc_data": "true", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_crc_header": "true", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T10:34:29.764 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_bug": "true", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_old_message": "true", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_hugepages": "", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T10:34:29.765 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_lro": "true", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_pmd": "", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_port_id": "0", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dump_on_send": "false", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_delay_probability": "0.005000", 2024-09-13T10:34:29.766 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_delay_type": "osd", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_internal_delays": "0.002000", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_network_congestion": "0", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_socket_failures": "2500", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_max_accept_failures": "4", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_max_backoff": "15.000000", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_osd_compress_mode": "none", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T10:34:29.767 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_pq_min_cost": "65536", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_public_type": "", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_service_mode": "crc secure", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_nodelay": "true", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_type": "async+posix", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "no_config_file": "false", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T10:34:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_inflight_ops": "1024", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_timeout": "10.000000", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objectstore_blackhole": "false", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "openssl_engine_opts": "", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T10:34:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_max_ops": "4", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_slop": "0.020000", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_asio_thread_count": "2", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backfill_scan_max": "512", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backfill_scan_min": "64", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backoff_on_peering": "false", 2024-09-13T10:34:29.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_beacon_report_interval": "300", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_duration": "30", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_blkin_trace_all": "false", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T10:34:29.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_update_on_start": "true", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_message_cap": "256", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_op_priority": "63", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_watch_timeout": "30", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_command_max_records": "256", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_compact_on_start": "false", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_update_on_start": "true", 2024-09-13T10:34:29.772 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_op_order": "false", 2024-09-13T10:34:29.773 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_shutdown": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_trim_objects": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T10:34:29.774 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_default_notify_timeout": "30", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_ec_partial_reads": "true", 2024-09-13T10:34:29.775 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_enable_op_tracker": "true", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_info": "true", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_shutdown": "true", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T10:34:29.776 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_function_tracing": "false", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_grace": "20", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_interval": "6", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_stale": "600", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T10:34:29.777 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_journal_size": "5120", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_kill_backfill_at": "0", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_cache_size": "50", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_dedup": "true", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_message_max": "40", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_attr_name_len": "100", 2024-09-13T10:34:29.778 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_attr_size": "0", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_backfills": "1", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_markdown_count": "5", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_markdown_period": "600", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_object_name_len": "2048", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_object_size": "134217728", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pgls": "1024", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_push_cost": "8388608", 2024-09-13T10:34:29.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_push_objects": "10", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_scrubs": "3", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_write_size": "90", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T10:34:29.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_profile": "balanced", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_base": "805306368", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T10:34:29.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_target": "4294967296", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_target_autotune": "false", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_min_recovery_priority": "0", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_report_interval": "5", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_num_cache_shards": "32", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T10:34:29.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_numa_node": "-1", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objecter_finishers": "1", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore": "bluestore", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore_fuse": "false", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore_tracing": "false", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_duration": "600", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_size": "20", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_log_threshold": "5", 2024-09-13T10:34:29.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_shards": "0", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_thread_timeout": "15", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_open_classes_on_start": "true", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_os_flags": "0", 2024-09-13T10:34:29.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_peering_op_priority": "255", 2024-09-13T10:34:29.785 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T10:34:29.785 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:29 smithi173 ceph-mon[30864]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_delete_priority": "5", 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T10:34:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_crimson": "false", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T10:34:29.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flags": "0", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_min_size": "2", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_read_ratio": "70", 2024-09-13T10:34:29.788 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_size": "2", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_type": "replicated", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_cost": "20971520", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_active": "0", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T10:34:29.789 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_op_priority": "3", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_priority": "5", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_repair_during_recovery": "false", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T10:34:29.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_cost": "52428800", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T10:34:29.791 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_end_hour": "0", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_priority": "5", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T10:34:29.792 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_skip_data_digest": "false", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_smart_report_timeout": "5", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_priority": "5", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T10:34:29.793 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_target_transaction_size": "30", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tracing": "false", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_use_stale_snap": "false", 2024-09-13T10:34:29.794 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_kill_at": "0", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_max_join_drift": "10", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_min": "500", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_min_wait": "0.050000", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_service_trim_max": "500", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_service_trim_min": "10", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_stash_full_interval": "25", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_trim_max": "500", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_trim_min": "250", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "perf": "true", 2024-09-13T10:34:29.795 INFO:teuthology.orchestra.run.smithi060.stdout: "pid_file": "", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "-", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrv": "", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "public_bind_addr": "-", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "public_network": "", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "public_network_interface": "", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "qat_compressor_busy_polling": "false", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "qat_compressor_enabled": "false", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "rados_mon_op_timeout": "0", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "rados_osd_op_timeout": "0", 2024-09-13T10:34:29.796 INFO:teuthology.orchestra.run.smithi060.stdout: "rados_tracing": "false", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_atime_update_interval": "60", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache": "true", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T10:34:29.797 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_policy": "writearound", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_size": "33554432", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_compression_hint": "none", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_clone_format": "auto", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_data_pool": "", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_features": "61", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_format": "2", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_map_options": "", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_order": "22", 2024-09-13T10:34:29.798 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_pool": "rbd", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_stripe_count": "0", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_io_scheduler": "simple", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T10:34:29.799 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_order": "24", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_pool": "", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_splay_width": "4", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T10:34:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T10:34:29.801 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T10:34:29.802 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_op_threads": "1", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_plugins": "", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T10:34:29.803 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T10:34:29.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_tracing": "false", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_validate_names": "true", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_validate_pool": "true", 2024-09-13T10:34:29.805 INFO:teuthology.orchestra.run.smithi060.stdout: "restapi_base_url": "", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "restapi_log_level": "", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_admin_entry": "admin", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_asio_assert_yielding": "false", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_backend_store": "rados", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_barbican_url": "", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_beast_enable_async": "true", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_counters_cache": "false", 2024-09-13T10:34:29.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cache_enabled": "true", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cache_lru_size": "25000", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_config_store": "rados", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_content_length_compat": "false", 2024-09-13T10:34:29.807 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cross_domain_policy": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T10:34:29.808 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T10:34:29.809 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T10:34:29.810 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T10:34:29.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T10:34:29.812 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_window": "30", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T10:34:29.813 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_disable_s3select": "false", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T10:34:29.814 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dns_name": "", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dns_s3website_name": "", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_mdsearch": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_ops_log": "false", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_static_website": "false", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_usage_log": "false", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T10:34:29.815 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_expose_bucket": "false", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_extended_http_attrs": "", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_filter": "none", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_objs": "32", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T10:34:29.816 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_init_timeout": "300", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_kafka_connection_idle": "300", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-13T10:34:29.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_password": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_project": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_token": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_user": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T10:34:29.818 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_url": "", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_objs": "32", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_worker": "3", 2024-09-13T10:34:29.819 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_uri": "", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T10:34:29.820 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_http_headers": "", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_attr_size": "0", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T10:34:29.821 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_listing_results": "1000", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_notify_retries": "10", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T10:34:29.822 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T10:34:29.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_num_control_oids": "8", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_numa_node": "-1", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T10:34:29.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_op_tracing": "false", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_opa_token": "", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_opa_url": "", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_rados": "false", 2024-09-13T10:34:29.825 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_lanes": "3", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-13T10:34:29.826 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_partitions": "3", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_print_continue": "true", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rados_tracing": "false", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_realm": "", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_realm_id": "", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T10:34:29.827 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_region": "", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_request_uri": "", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_resolve_cname": "false", 2024-09-13T10:34:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_run_sync_thread": "true", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_script_uri": "", 2024-09-13T10:34:29.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_service_provider_name": "", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_client_id": "", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_client_secret": "", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_entry": "sts", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_key": "sts", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_auth_url": "", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_custom_header": "", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T10:34:29.830 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_need_stats": "true", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_tenant_name": "", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_url": "", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_lease_period": "120", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T10:34:29.831 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_thread_pool_size": "512", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-13T10:34:29.832 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_comment": "", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_createby": "", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_encoding": "", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_flag": "false", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_origin": "", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_tracker": "", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_max_shards": "32", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T10:34:29.833 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_use_opa_authz": "false", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_counters_cache": "false", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_unique_email": "true", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_verify_ssl": "true", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T10:34:29.834 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zone": "", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zone_id": "", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zonegroup": "", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zonegroup_id": "", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_block_size": "4096", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T10:34:29.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_partition_filters": "false", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_perf": "false", 2024-09-13T10:34:29.836 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "run_dir": "/var/run/ceph", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_block_create": "true", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_device_size": "53687091200", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_full_integrity_check": "false", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T10:34:29.837 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_main_device_type": "SSD", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_segment_size": "67108864", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "set_keepcaps": "false", 2024-09-13T10:34:29.838 INFO:teuthology.orchestra.run.smithi060.stdout: "setgroup": "ceph", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "setuser": "ceph", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "setuser_match_path": "", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "thp": "false", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "threadpool_default_timeout": "60", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "throttler_perf_counter": "true", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tmp_dir": "/tmp", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "uadk_compressor_enabled": "false", 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-13T10:34:29.839 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:34:29.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:29 smithi138 ceph-mon[31033]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:30.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T10:34:31.075 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:31.521 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a: {} 2024-09-13T10:34:31.521 INFO:teuthology.orchestra.run.smithi060.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:34:31.537 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c: {} 2024-09-13T10:34:31.538 INFO:teuthology.orchestra.run.smithi060.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:34:31.552 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b: {} 2024-09-13T10:34:31.552 INFO:teuthology.orchestra.run.smithi060.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:34:31.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:31 smithi173 ceph-mon[30864]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:31.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:31 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:31.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:31 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:31.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:31 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3139797327' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:34:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:31 smithi138 ceph-mon[31033]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:31 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:31 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:31.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:31 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3139797327' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:34:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:31 smithi060 ceph-mon[27375]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:31 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:31 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:31.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:31 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3139797327' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:34:32.266 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T10:34:32.276 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T10:34:32.276 INFO:teuthology.task.exec:Running commands on role client.2 host ubuntu@smithi173.front.sepia.ceph.com 2024-09-13T10:34:32.276 DEBUG:teuthology.orchestra.run.smithi173:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror' 2024-09-13T10:34:32.277 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T10:34:32.277 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-13T10:34:32.277 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:34:32.277 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T10:34:32.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-13T10:34:32.278 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T10:34:32.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:32.278 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T10:34:32.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:34:32.279 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T10:34:32.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-13T10:34:32.279 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T10:34:32.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-13T10:34:32.312 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:32.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:32.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:32.325 INFO:teuthology.orchestra.run.smithi173.stdout:Running main() from gmock_main.cc 2024-09-13T10:34:32.325 INFO:teuthology.orchestra.run.smithi173.stdout:Note: Google Test filter = -TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror 2024-09-13T10:34:32.325 INFO:teuthology.orchestra.run.smithi173.stdout:[==========] Running 68 tests from 1 test suite. 2024-09-13T10:34:32.325 INFO:teuthology.orchestra.run.smithi173.stdout:[----------] Global test environment set-up. 2024-09-13T10:34:32.325 INFO:teuthology.orchestra.run.smithi173.stdout:[----------] 68 tests from TestClsRbd 2024-09-13T10:34:32.414 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:32.438 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:32.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:32.439 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:32.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:32.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:32.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:32.591 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:32.598 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:32.598 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:32.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:32.690 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:32.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:32.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:32.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:32 smithi173 ceph-mon[30864]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T10:34:32.791 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:32.817 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:32.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:32.818 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:32.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:32 smithi138 ceph-mon[31033]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T10:34:32.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:32 smithi060 ceph-mon[27375]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T10:34:32.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:32.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:32.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:33.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:33.077 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:33.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:33.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:33.201 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:33.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:33.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 97, 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "74376cb6-71ba-11ef-bceb-c7b262605968", 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-13T10:26:24.281921+0000", 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-13T10:34:32.527188+0000", 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-13T10:33:31.317557+0000", 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-13T10:33:11.111853+0000", 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5799936, 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-13T10:34:33.307 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 26, 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 2, 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 12, 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-13T10:34:33.308 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "jewel", 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-13T10:29:06.355752+0000", 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-13T10:34:33.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 2, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 1, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 1, 2024-09-13T10:34:33.310 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 1, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 1, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 1, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "17", 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-13T10:34:33.311 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-13T10:34:33.312 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.313 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-13T10:34:33.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 11.760000228881836, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 11.760000228881836, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.17000000178813934, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.99999988079071045, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-13T10:34:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "test-rados-api-smithi173-56683-1", 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-13T10:34:32.350056+0000", 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 32769, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,creating", 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 2, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-13T10:34:33.316 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-13T10:34:33.317 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "97", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-13T10:34:33.318 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:34:33.319 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:34:33.320 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": {}, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.8799999952316284, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.8799999952316284, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-13T10:34:33.321 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.99999988079071045, 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "3db36d02-fea0-4b18-8994-97158486c7cd", 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.322 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 52, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.325 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-13T10:34:33.325 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-13T10:34:33.326 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.327 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 756062006 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/756062006", 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/756062006", 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/756062006", 2024-09-13T10:34:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/756062006", 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "f2323c13-004c-47e1-81ed-44e6c9fb193a", 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.329 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 13, 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 15, 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3814817161 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/3814817161", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/3814817161", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/3814817161", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/3814817161", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.334 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "bda5f20f-8413-4a97-9969-1eea2f1f7b87", 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 21, 2024-09-13T10:34:33.335 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-13T10:34:33.336 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-13T10:34:33.337 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.338 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1235426811 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/1235426811", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/1235426811", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/1235426811", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/1235426811", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "43cba2dc-ab85-4d9a-9b2e-c4fd527077a0", 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 27, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.341 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.342 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-13T10:34:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3263875168 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/3263875168", 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/3263875168", 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/3263875168", 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/3263875168", 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b76daf81-be0c-42b5-b262-a4a0cfd615f8", 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 33, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.347 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6800", 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6801", 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6802", 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6803", 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6806", 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6807", 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6804", 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6805", 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2143931671 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.351 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.138:6801/2143931671", 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.138:6803/2143931671", 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.138:6807/2143931671", 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.138:6805/2143931671", 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "9037c1e7-d476-4980-8155-6244430c9011", 2024-09-13T10:34:33.352 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 39, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6808", 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6809", 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6810", 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6811", 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6814", 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6815", 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6812", 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6813", 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2632797914 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.357 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.138:6809/2632797914", 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.138:6811/2632797914", 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.138:6815/2632797914", 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.138:6813/2632797914", 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b635470b-4619-4a8a-b016-1811d5049544", 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 46, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 47, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6816", 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6817", 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6818", 2024-09-13T10:34:33.360 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6819", 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6822", 2024-09-13T10:34:33.361 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6823", 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6820", 2024-09-13T10:34:33.362 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6821", 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3026996312 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.138:6817/3026996312", 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.138:6819/3026996312", 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.138:6823/3026996312", 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.138:6821/3026996312", 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.363 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4c58cdb7-9705-4a68-816f-8d6ddeca34d3", 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.364 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 52, 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 85, 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6824", 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.365 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6825", 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6826", 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.366 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6827", 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6830", 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.367 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6831", 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6828", 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.138:6829", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1381162744 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.138:6825/1381162744", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.138:6827/1381162744", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.138:6831/1381162744", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.138:6829/1381162744", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 8, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b77d95ef-5b65-4446-ae22-ef2dfe094667", 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 61, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.370 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6800", 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6801", 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6802", 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6803", 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.372 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6806", 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6807", 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6804", 2024-09-13T10:34:33.374 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6805", 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1825049558 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.376 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.173:6801/1825049558", 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.173:6803/1825049558", 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.173:6807/1825049558", 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.173:6805/1825049558", 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 9, 2024-09-13T10:34:33.377 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b89d58eb-56f0-4ede-aa1b-8d6c25be30af", 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 69, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6808", 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6809", 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.379 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6810", 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6811", 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.380 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6814", 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6815", 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.381 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6812", 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6813", 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 417733706 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.382 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.173:6809/417733706", 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.173:6811/417733706", 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.173:6815/417733706", 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.173:6813/417733706", 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 10, 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "46ddc07a-0a77-445e-bc19-a345e2cdce5d", 2024-09-13T10:34:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 77, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6816", 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6817", 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6818", 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6819", 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6822", 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6823", 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6820", 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6821", 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3542783063 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.173:6817/3542783063", 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.173:6819/3542783063", 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.173:6823/3542783063", 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.173:6821/3542783063", 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 11, 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "6476cc59-b51a-43a8-962e-1ae48119c362", 2024-09-13T10:34:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 84, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 0, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6824", 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6825", 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6826", 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6827", 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6830", 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6831", 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6828", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.173:6829", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4131510497 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.173:6825/4131510497", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.173:6827/4131510497", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.173:6831/4131510497", 2024-09-13T10:34:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.173:6829/4131510497", 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:33:37.337476+0000", 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:33:41.844672+0000", 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:34:19.821695+0000", 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:33:30.349970+0000", 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:34:16.032652+0000", 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:34:31.074089+0000", 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:33:45.526672+0000", 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:34:10.874606+0000", 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 8, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:34:23.980012+0000", 2024-09-13T10:34:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 9, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:33:56.058565+0000", 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 10, 2024-09-13T10:34:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:34:21.861878+0000", 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 11, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540719139924082687, 2024-09-13T10:34:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-13T10:33:29.765854+0000", 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [], 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [], 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [], 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2156160235": "2024-09-14T10:27:11.894382+0000", 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1488046566": "2024-09-14T10:27:11.894382+0000", 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/250706696": "2024-09-14T10:27:11.894382+0000", 2024-09-13T10:34:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/250706696": "2024-09-14T10:27:11.894382+0000", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2405347332": "2024-09-14T10:27:11.894382+0000", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1860079602": "2024-09-14T10:26:47.425146+0000", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3783615351": "2024-09-14T10:26:47.425146+0000", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3783615351": "2024-09-14T10:26:47.425146+0000", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2669724212": "2024-09-14T10:26:47.425146+0000", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1079154737": "2024-09-14T10:26:47.425146+0000" 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-13T10:34:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-13T10:34:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:33.408 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:34:33.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.408 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:33.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:33.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:33.536 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:33.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.537 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:33.540 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_all_features 2024-09-13T10:34:33.550 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_all_features (10 ms) 2024-09-13T10:34:33.550 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.copyup 2024-09-13T10:34:33.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:33 smithi060 ceph-mon[27375]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:33.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:33 smithi060 ceph-mon[27375]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T10:34:33.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:33 smithi060 ceph-mon[27375]: from='client.? 172.21.15.173:0/1369318498' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T10:34:33.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:33 smithi060 ceph-mon[27375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T10:34:33.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:33 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4238006740' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-13T10:34:33.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:33.664 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:33.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:33.716 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.copyup (166 ms) 2024-09-13T10:34:33.716 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.sparse_copyup 2024-09-13T10:34:33.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:33.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:33.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:33 smithi138 ceph-mon[31033]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:33.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:33 smithi138 ceph-mon[31033]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T10:34:33.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:33 smithi138 ceph-mon[31033]: from='client.? 172.21.15.173:0/1369318498' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T10:34:33.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:33 smithi138 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T10:34:33.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:33 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4238006740' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-13T10:34:33.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:33.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:33.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:34.014 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:34.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:33 smithi173 ceph-mon[30864]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:34.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:33 smithi173 ceph-mon[30864]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T10:34:34.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:33 smithi173 ceph-mon[30864]: from='client.? 172.21.15.173:0/1369318498' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T10:34:34.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:33 smithi173 ceph-mon[30864]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-13T10:34:34.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:33 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4238006740' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-13T10:34:34.040 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:34.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.041 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:34.131 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.sparse_copyup (414 ms) 2024-09-13T10:34:34.132 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_and_set_id 2024-09-13T10:34:34.142 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:34.146 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_and_set_id (14 ms) 2024-09-13T10:34:34.146 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.add_remove_child 2024-09-13T10:34:34.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.166 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:34.167 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:34.168 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.add_remove_child (24 ms) 2024-09-13T10:34:34.168 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.directory_methods 2024-09-13T10:34:34.200 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.directory_methods (31 ms) 2024-09-13T10:34:34.201 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.create 2024-09-13T10:34:34.219 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.create (20 ms) 2024-09-13T10:34:34.220 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_object_prefix 2024-09-13T10:34:34.222 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_object_prefix (2 ms) 2024-09-13T10:34:34.222 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_create_timestamp 2024-09-13T10:34:34.223 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_create_timestamp (2 ms) 2024-09-13T10:34:34.223 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_access_timestamp 2024-09-13T10:34:34.230 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_access_timestamp (6 ms) 2024-09-13T10:34:34.233 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_modify_timestamp 2024-09-13T10:34:34.236 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_modify_timestamp (6 ms) 2024-09-13T10:34:34.236 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_data_pool 2024-09-13T10:34:34.240 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_data_pool (4 ms) 2024-09-13T10:34:34.240 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.get_size 2024-09-13T10:34:34.250 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.get_size (10 ms) 2024-09-13T10:34:34.250 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.set_size 2024-09-13T10:34:34.260 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.set_size (10 ms) 2024-09-13T10:34:34.260 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.protection_status 2024-09-13T10:34:34.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:34.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.292 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:34.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:34.394 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:34.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.417 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:34.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:34.503 INFO:tasks.thrashosds.thrasher:j is {'epoch': 97, 'fsid': '74376cb6-71ba-11ef-bceb-c7b262605968', 'created': '2024-09-13T10:26:24.281921+0000', 'modified': '2024-09-13T10:34:32.527188+0000', 'last_up_change': '2024-09-13T10:33:31.317557+0000', 'last_in_change': '2024-09-13T10:33:11.111853+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 26, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:29:06.355752+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 11.760000228881836, 'score_stable': 11.760000228881836, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'test-rados-api-smithi173-56683-1', 'create_time': '2024-09-13T10:34:32.350056+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3db36d02-fea0-4b18-8994-97158486c7cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 756062006}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 756062006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 756062006}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 756062006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 756062006}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 756062006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 756062006}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 756062006}]}, 'public_addr': '172.21.15.60:6803/756062006', 'cluster_addr': '172.21.15.60:6805/756062006', 'heartbeat_back_addr': '172.21.15.60:6809/756062006', 'heartbeat_front_addr': '172.21.15.60:6807/756062006', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f2323c13-004c-47e1-81ed-44e6c9fb193a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 3814817161}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 3814817161}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 3814817161}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 3814817161}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 3814817161}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 3814817161}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 3814817161}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 3814817161}]}, 'public_addr': '172.21.15.60:6811/3814817161', 'cluster_addr': '172.21.15.60:6813/3814817161', 'heartbeat_back_addr': '172.21.15.60:6817/3814817161', 'heartbeat_front_addr': '172.21.15.60:6815/3814817161', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bda5f20f-8413-4a97-9969-1eea2f1f7b87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1235426811}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 1235426811}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 1235426811}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 1235426811}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 1235426811}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 1235426811}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 1235426811}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 1235426811}]}, 'public_addr': '172.21.15.60:6819/1235426811', 'cluster_addr': '172.21.15.60:6821/1235426811', 'heartbeat_back_addr': '172.21.15.60:6825/1235426811', 'heartbeat_front_addr': '172.21.15.60:6823/1235426811', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '43cba2dc-ab85-4d9a-9b2e-c4fd527077a0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 3263875168}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 3263875168}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 3263875168}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 3263875168}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 3263875168}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 3263875168}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 3263875168}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 3263875168}]}, 'public_addr': '172.21.15.60:6827/3263875168', 'cluster_addr': '172.21.15.60:6829/3263875168', 'heartbeat_back_addr': '172.21.15.60:6833/3263875168', 'heartbeat_front_addr': '172.21.15.60:6831/3263875168', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b76daf81-be0c-42b5-b262-a4a0cfd615f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 2143931671}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 2143931671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 2143931671}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 2143931671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 2143931671}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 2143931671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 2143931671}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 2143931671}]}, 'public_addr': '172.21.15.138:6801/2143931671', 'cluster_addr': '172.21.15.138:6803/2143931671', 'heartbeat_back_addr': '172.21.15.138:6807/2143931671', 'heartbeat_front_addr': '172.21.15.138:6805/2143931671', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '9037c1e7-d476-4980-8155-6244430c9011', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6808', 'nonce': 2632797914}, {'type': 'v1', 'addr': '172.21.15.138:6809', 'nonce': 2632797914}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 2632797914}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 2632797914}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 2632797914}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 2632797914}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 2632797914}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 2632797914}]}, 'public_addr': '172.21.15.138:6809/2632797914', 'cluster_addr': '172.21.15.138:6811/2632797914', 'heartbeat_back_addr': '172.21.15.138:6815/2632797914', 'heartbeat_front_addr': '172.21.15.138:6813/2632797914', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'b635470b-4619-4a8a-b016-1811d5049544', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 3026996312}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 3026996312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 3026996312}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 3026996312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 3026996312}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 3026996312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 3026996312}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 3026996312}]}, 'public_addr': '172.21.15.138:6817/3026996312', 'cluster_addr': '172.21.15.138:6819/3026996312', 'heartbeat_back_addr': '172.21.15.138:6823/3026996312', 'heartbeat_front_addr': '172.21.15.138:6821/3026996312', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4c58cdb7-9705-4a68-816f-8d6ddeca34d3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 1381162744}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 1381162744}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1381162744}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1381162744}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1381162744}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1381162744}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1381162744}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1381162744}]}, 'public_addr': '172.21.15.138:6825/1381162744', 'cluster_addr': '172.21.15.138:6827/1381162744', 'heartbeat_back_addr': '172.21.15.138:6831/1381162744', 'heartbeat_front_addr': '172.21.15.138:6829/1381162744', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'b77d95ef-5b65-4446-ae22-ef2dfe094667', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 61, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6800', 'nonce': 1825049558}, {'type': 'v1', 'addr': '172.21.15.173:6801', 'nonce': 1825049558}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6802', 'nonce': 1825049558}, {'type': 'v1', 'addr': '172.21.15.173:6803', 'nonce': 1825049558}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6806', 'nonce': 1825049558}, {'type': 'v1', 'addr': '172.21.15.173:6807', 'nonce': 1825049558}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6804', 'nonce': 1825049558}, {'type': 'v1', 'addr': '172.21.15.173:6805', 'nonce': 1825049558}]}, 'public_addr': '172.21.15.173:6801/1825049558', 'cluster_addr': '172.21.15.173:6803/1825049558', 'heartbeat_back_addr': '172.21.15.173:6807/1825049558', 'heartbeat_front_addr': '172.21.15.173:6805/1825049558', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'b89d58eb-56f0-4ede-aa1b-8d6c25be30af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 69, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6808', 'nonce': 417733706}, {'type': 'v1', 'addr': '172.21.15.173:6809', 'nonce': 417733706}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6810', 'nonce': 417733706}, {'type': 'v1', 'addr': '172.21.15.173:6811', 'nonce': 417733706}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6814', 'nonce': 417733706}, {'type': 'v1', 'addr': '172.21.15.173:6815', 'nonce': 417733706}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6812', 'nonce': 417733706}, {'type': 'v1', 'addr': '172.21.15.173:6813', 'nonce': 417733706}]}, 'public_addr': '172.21.15.173:6809/417733706', 'cluster_addr': '172.21.15.173:6811/417733706', 'heartbeat_back_addr': '172.21.15.173:6815/417733706', 'heartbeat_front_addr': '172.21.15.173:6813/417733706', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '46ddc07a-0a77-445e-bc19-a345e2cdce5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 77, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6816', 'nonce': 3542783063}, {'type': 'v1', 'addr': '172.21.15.173:6817', 'nonce': 3542783063}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6818', 'nonce': 3542783063}, {'type': 'v1', 'addr': '172.21.15.173:6819', 'nonce': 3542783063}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6822', 'nonce': 3542783063}, {'type': 'v1', 'addr': '172.21.15.173:6823', 'nonce': 3542783063}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6820', 'nonce': 3542783063}, {'type': 'v1', 'addr': '172.21.15.173:6821', 'nonce': 3542783063}]}, 'public_addr': '172.21.15.173:6817/3542783063', 'cluster_addr': '172.21.15.173:6819/3542783063', 'heartbeat_back_addr': '172.21.15.173:6823/3542783063', 'heartbeat_front_addr': '172.21.15.173:6821/3542783063', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '6476cc59-b51a-43a8-962e-1ae48119c362', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 84, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6824', 'nonce': 4131510497}, {'type': 'v1', 'addr': '172.21.15.173:6825', 'nonce': 4131510497}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6826', 'nonce': 4131510497}, {'type': 'v1', 'addr': '172.21.15.173:6827', 'nonce': 4131510497}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6830', 'nonce': 4131510497}, {'type': 'v1', 'addr': '172.21.15.173:6831', 'nonce': 4131510497}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6828', 'nonce': 4131510497}, {'type': 'v1', 'addr': '172.21.15.173:6829', 'nonce': 4131510497}]}, 'public_addr': '172.21.15.173:6825/4131510497', 'cluster_addr': '172.21.15.173:6827/4131510497', 'heartbeat_back_addr': '172.21.15.173:6831/4131510497', 'heartbeat_front_addr': '172.21.15.173:6829/4131510497', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:33:37.337476+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:33:41.844672+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:34:19.821695+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:33:30.349970+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:34:16.032652+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:34:31.074089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:33:45.526672+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:34:10.874606+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:34:23.980012+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:33:56.058565+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:34:21.861878+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:33:29.765854+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/2156160235': '2024-09-14T10:27:11.894382+0000', '172.21.15.60:0/1488046566': '2024-09-14T10:27:11.894382+0000', '172.21.15.60:6801/250706696': '2024-09-14T10:27:11.894382+0000', '172.21.15.60:6800/250706696': '2024-09-14T10:27:11.894382+0000', '172.21.15.60:0/2405347332': '2024-09-14T10:27:11.894382+0000', '172.21.15.60:0/1860079602': '2024-09-14T10:26:47.425146+0000', '172.21.15.60:6801/3783615351': '2024-09-14T10:26:47.425146+0000', '172.21.15.60:6800/3783615351': '2024-09-14T10:26:47.425146+0000', '172.21.15.60:0/2669724212': '2024-09-14T10:26:47.425146+0000', '172.21.15.60:0/1079154737': '2024-09-14T10:26:47.425146+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T10:34:34.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:34:34.519 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:34.541 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:34.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.543 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:34.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:34.667 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:34.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:34.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:34 smithi060 ceph-mon[27375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T10:34:34.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:34 smithi060 ceph-mon[27375]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T10:34:34.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:34 smithi060 ceph-mon[27375]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T10:34:34.747 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:34:34.748 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T10:34:34.748 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T10:34:34.748 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:34:34.769 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:34.792 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:34.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.793 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:34.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:34 smithi138 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T10:34:34.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:34 smithi138 ceph-mon[31033]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T10:34:34.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:34 smithi138 ceph-mon[31033]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T10:34:34.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:34.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:34.918 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:34.918 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:35.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:35.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:34 smithi173 ceph-mon[30864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi173-56683-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-13T10:34:35.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:34 smithi173 ceph-mon[30864]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T10:34:35.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:34 smithi173 ceph-mon[30864]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T10:34:35.045 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:35.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:35.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:35.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:35.267 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.protection_status (1007 ms) 2024-09-13T10:34:35.267 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.snapshot_limits 2024-09-13T10:34:35.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:35.275 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.snapshot_limits (8 ms) 2024-09-13T10:34:35.275 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.parents_v1 2024-09-13T10:34:35.294 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:35.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:35.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:35.418 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:35.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.420 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:35.521 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:35.547 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:35.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:35.563 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-13T10:34:35.563 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:34:35.584 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-13T10:34:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:34:35.604 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-13T10:34:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:34:35.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:35 smithi060 ceph-mon[27375]: pgmap v309: 33 pgs: 3 active+clean, 12 creating+peering, 18 unknown; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:35.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:35 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1370371973' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:35.625 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-13T10:34:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:34:35.646 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-13T10:34:35.647 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:34:35.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:35.668 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-13T10:34:35.668 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:34:35.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.675 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:35.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:35.689 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-13T10:34:35.689 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:34:35.711 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-13T10:34:35.711 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:34:35.733 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8: {} 2024-09-13T10:34:35.733 INFO:teuthology.orchestra.run.smithi060.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:34:35.754 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9: {} 2024-09-13T10:34:35.754 INFO:teuthology.orchestra.run.smithi060.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:34:35.775 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10: {} 2024-09-13T10:34:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:34:35.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:35.796 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11: {} 2024-09-13T10:34:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:34:35.798 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:35.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:35.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:35 smithi138 ceph-mon[31033]: pgmap v309: 33 pgs: 3 active+clean, 12 creating+peering, 18 unknown; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:35.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:35 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1370371973' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:35.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:35.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:35.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:36.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:35 smithi173 ceph-mon[30864]: pgmap v309: 33 pgs: 3 active+clean, 12 creating+peering, 18 unknown; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:34:36.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:35 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1370371973' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:36.028 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:36.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-13T10:34:36.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.052 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:36.052 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:36.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:36.176 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:36.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.177 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:36.278 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:36.300 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:36.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.301 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:36.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:36.423 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:36.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.424 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:36.525 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:36.546 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:36.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.548 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:36.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:36.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.674 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:36.674 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:36.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:36.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:36.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:36.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:36.926 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:36.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:36.933 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.parents_v1 (1658 ms) 2024-09-13T10:34:36.933 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.parents_v2 2024-09-13T10:34:37.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:37.033 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.parents_v2 (101 ms) 2024-09-13T10:34:37.033 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.snapshots 2024-09-13T10:34:37.048 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:37.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.049 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:37.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:37.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:37.274 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:37.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.297 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:37.297 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:37.398 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:37.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.420 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:37.420 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:37.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:37.542 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:37.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.543 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:37.548 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.snapshots (515 ms) 2024-09-13T10:34:37.548 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.snapid_race 2024-09-13T10:34:37.555 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.snapid_race (6 ms) 2024-09-13T10:34:37.555 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.stripingv2 2024-09-13T10:34:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-13T10:34:37.568 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.stripingv2 (13 ms) 2024-09-13T10:34:37.568 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.object_map_save 2024-09-13T10:34:37.571 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.object_map_save (4 ms) 2024-09-13T10:34:37.571 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.object_map_resize 2024-09-13T10:34:37.580 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.object_map_resize (8 ms) 2024-09-13T10:34:37.580 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.object_map_update 2024-09-13T10:34:37.587 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.object_map_update (7 ms) 2024-09-13T10:34:37.587 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.object_map_load_enoent 2024-09-13T10:34:37.587 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.object_map_load_enoent (0 ms) 2024-09-13T10:34:37.587 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.object_map_snap_add 2024-09-13T10:34:37.594 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.object_map_snap_add (8 ms) 2024-09-13T10:34:37.594 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.object_map_snap_remove 2024-09-13T10:34:37.602 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.object_map_snap_remove (7 ms) 2024-09-13T10:34:37.602 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.flags 2024-09-13T10:34:37.606 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.flags (5 ms) 2024-09-13T10:34:37.606 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.metadata 2024-09-13T10:34:37.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:37 smithi060 ceph-mon[27375]: pgmap v311: 33 pgs: 11 active+clean, 12 creating+peering, 10 unknown; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2024-09-13T10:34:37.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:37 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2527547342' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-13T10:34:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.metadata (29 ms) 2024-09-13T10:34:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.set_features 2024-09-13T10:34:37.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.set_features (8 ms) 2024-09-13T10:34:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.mirror_image 2024-09-13T10:34:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.mirror_image (18 ms) 2024-09-13T10:34:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.mirror_image_status 2024-09-13T10:34:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:37.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:37.771 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:37.793 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:37.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:37.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:37 smithi138 ceph-mon[31033]: pgmap v311: 33 pgs: 11 active+clean, 12 creating+peering, 10 unknown; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2024-09-13T10:34:37.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:37 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2527547342' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-13T10:34:37.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:37.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:37.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:37.931 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:34:38.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:38.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:37 smithi173 ceph-mon[30864]: pgmap v311: 33 pgs: 11 active+clean, 12 creating+peering, 10 unknown; 577 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 KiB/s wr, 17 op/s 2024-09-13T10:34:38.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:37 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2527547342' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-13T10:34:38.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.044 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:38.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:38.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:38.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.170 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:38.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:38.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:38.294 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:38.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:38.396 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:38.419 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:38.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.420 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:38.521 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:38.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:38.605 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-13T10:34:37.555212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590388,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":316,"num_read_kb":282,"num_write":185,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":11,"kb":1124941824,"kb_used":338596,"kb_used_data":10056,"kb_used_omap":18,"kb_used_meta":328237,"kb_avail":1124603228,"statfs":{"total":1151940427776,"available":1151593705472,"internally_reserved":0,"allocated":10297344,"data_stored":5920591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":336115002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":240,"num_read_kb":218,"num_write":62,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.578187"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.539894+0000","last_change":"2024-09-13T10:34:33.543301+0000","last_active":"2024-09-13T10:34:35.539894+0000","last_peered":"2024-09-13T10:34:35.539894+0000","last_clean":"2024-09-13T10:34:35.539894+0000","last_became_active":"2024-09-13T10:34:33.543049+0000","last_became_peered":"2024-09-13T10:34:33.543049+0000","last_unstale":"2024-09-13T10:34:35.539894+0000","last_undegraded":"2024-09-13T10:34:35.539894+0000","last_fullsized":"2024-09-13T10:34:35.539894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:59.943935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"98'7","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:34.538125+0000","last_change":"2024-09-13T10:34:33.545736+0000","last_active":"2024-09-13T10:34:34.538125+0000","last_peered":"2024-09-13T10:34:34.538125+0000","last_clean":"2024-09-13T10:34:34.538125+0000","last_became_active":"2024-09-13T10:34:33.545495+0000","last_became_peered":"2024-09-13T10:34:33.545495+0000","last_unstale":"2024-09-13T10:34:34.538125+0000","last_undegraded":"2024-09-13T10:34:34.538125+0000","last_fullsized":"2024-09-13T10:34:34.538125+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:38.470069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540565+0000","last_change":"2024-09-13T10:34:32.540565+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540565+0000","last_undegraded":"2024-09-13T10:34:32.540565+0000","last_fullsized":"2024-09-13T10:34:32.540565+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7],"acting":[10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540547+0000","last_change":"2024-09-13T10:34:32.540547+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540547+0000","last_undegraded":"2024-09-13T10:34:32.540547+0000","last_fullsized":"2024-09-13T10:34:32.540547+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7],"acting":[10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.535487+0000","last_change":"2024-09-13T10:34:32.535487+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.535487+0000","last_undegraded":"2024-09-13T10:34:32.535487+0000","last_fullsized":"2024-09-13T10:34:32.535487+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.a","version":"99'2","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.247361+0000","last_change":"2024-09-13T10:34:33.544265+0000","last_active":"2024-09-13T10:34:35.247361+0000","last_peered":"2024-09-13T10:34:35.247361+0000","last_clean":"2024-09-13T10:34:35.247361+0000","last_became_active":"2024-09-13T10:34:33.544016+0000","last_became_peered":"2024-09-13T10:34:33.544016+0000","last_unstale":"2024-09-13T10:34:35.247361+0000","last_undegraded":"2024-09-13T10:34:35.247361+0000","last_fullsized":"2024-09-13T10:34:35.247361+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:32.957770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3],"acting":[11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-13T10:34:33.546027+0000","last_change":"2024-09-13T10:34:33.545215+0000","last_active":"2024-09-13T10:34:33.546027+0000","last_peered":"2024-09-13T10:34:33.546027+0000","last_clean":"2024-09-13T10:34:33.546027+0000","last_became_active":"2024-09-13T10:34:33.544543+0000","last_became_peered":"2024-09-13T10:34:33.544543+0000","last_unstale":"2024-09-13T10:34:33.546027+0000","last_undegraded":"2024-09-13T10:34:33.546027+0000","last_fullsized":"2024-09-13T10:34:33.546027+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:34.546752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-13T10:34:33.544538+0000","last_change":"2024-09-13T10:34:33.543946+0000","last_active":"2024-09-13T10:34:33.544538+0000","last_peered":"2024-09-13T10:34:33.544538+0000","last_clean":"2024-09-13T10:34:33.544538+0000","last_became_active":"2024-09-13T10:34:33.543672+0000","last_became_peered":"2024-09-13T10:34:33.543672+0000","last_unstale":"2024-09-13T10:34:33.544538+0000","last_undegraded":"2024-09-13T10:34:33.544538+0000","last_fullsized":"2024-09-13T10:34:33.544538+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:50.250233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540046+0000","last_change":"2024-09-13T10:34:32.540046+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540046+0000","last_undegraded":"2024-09-13T10:34:32.540046+0000","last_fullsized":"2024-09-13T10:34:32.540046+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"98'1","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-13T10:34:34.235611+0000","last_change":"2024-09-13T10:34:33.540613+0000","last_active":"2024-09-13T10:34:34.235611+0000","last_peered":"2024-09-13T10:34:34.235611+0000","last_clean":"2024-09-13T10:34:34.235611+0000","last_became_active":"2024-09-13T10:34:33.539414+0000","last_became_peered":"2024-09-13T10:34:33.539414+0000","last_unstale":"2024-09-13T10:34:34.235611+0000","last_undegraded":"2024-09-13T10:34:34.235611+0000","last_fullsized":"2024-09-13T10:34:34.235611+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:39.879479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.535486+0000","last_change":"2024-09-13T10:34:32.535486+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.535486+0000","last_undegraded":"2024-09-13T10:34:32.535486+0000","last_fullsized":"2024-09-13T10:34:32.535486+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-13T10:34:33.545992+0000","last_change":"2024-09-13T10:34:33.544792+0000","last_active":"2024-09-13T10:34:33.545992+0000","last_peered":"2024-09-13T10:34:33.545992+0000","last_clean":"2024-09-13T10:34:33.545992+0000","last_became_active":"2024-09-13T10:34:33.544466+0000","last_became_peered":"2024-09-13T10:34:33.544466+0000","last_unstale":"2024-09-13T10:34:33.545992+0000","last_undegraded":"2024-09-13T10:34:33.545992+0000","last_fullsized":"2024-09-13T10:34:33.545992+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:53.124593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:34.540019+0000","last_change":"2024-09-13T10:34:33.549230+0000","last_active":"2024-09-13T10:34:34.540019+0000","last_peered":"2024-09-13T10:34:34.540019+0000","last_clean":"2024-09-13T10:34:34.540019+0000","last_became_active":"2024-09-13T10:34:33.548580+0000","last_became_peered":"2024-09-13T10:34:33.548580+0000","last_unstale":"2024-09-13T10:34:34.540019+0000","last_undegraded":"2024-09-13T10:34:34.540019+0000","last_fullsized":"2024-09-13T10:34:34.540019+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:59.968990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'15","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:34.980189+0000","last_change":"2024-09-13T10:34:33.543767+0000","last_active":"2024-09-13T10:34:34.980189+0000","last_peered":"2024-09-13T10:34:34.980189+0000","last_clean":"2024-09-13T10:34:34.980189+0000","last_became_active":"2024-09-13T10:34:33.543529+0000","last_became_peered":"2024-09-13T10:34:33.543529+0000","last_unstale":"2024-09-13T10:34:34.980189+0000","last_undegraded":"2024-09-13T10:34:34.980189+0000","last_fullsized":"2024-09-13T10:34:34.980189+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:52.434490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-13T10:34:33.544510+0000","last_change":"2024-09-13T10:34:33.543759+0000","last_active":"2024-09-13T10:34:33.544510+0000","last_peered":"2024-09-13T10:34:33.544510+0000","last_clean":"2024-09-13T10:34:33.544510+0000","last_became_active":"2024-09-13T10:34:33.543288+0000","last_became_peered":"2024-09-13T10:34:33.543288+0000","last_unstale":"2024-09-13T10:34:33.544510+0000","last_undegraded":"2024-09-13T10:34:33.544510+0000","last_fullsized":"2024-09-13T10:34:33.544510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:41.222478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:34.540016+0000","last_change":"2024-09-13T10:34:33.546377+0000","last_active":"2024-09-13T10:34:34.540016+0000","last_peered":"2024-09-13T10:34:34.540016+0000","last_clean":"2024-09-13T10:34:34.540016+0000","last_became_active":"2024-09-13T10:34:33.546143+0000","last_became_peered":"2024-09-13T10:34:33.546143+0000","last_unstale":"2024-09-13T10:34:34.540016+0000","last_undegraded":"2024-09-13T10:34:34.540016+0000","last_fullsized":"2024-09-13T10:34:34.540016+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:43.182551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'92","reported_seq":147,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-13T10:34:33.541637+0000","last_change":"2024-09-13T10:34:13.003713+0000","last_active":"2024-09-13T10:34:33.541637+0000","last_peered":"2024-09-13T10:34:33.541637+0000","last_clean":"2024-09-13T10:34:33.541637+0000","last_became_active":"2024-09-13T10:33:33.765804+0000","last_became_peered":"2024-09-13T10:33:33.765804+0000","last_unstale":"2024-09-13T10:34:33.541637+0000","last_undegraded":"2024-09-13T10:34:33.541637+0000","last_fullsized":"2024-09-13T10:34:33.541637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:16.331942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.240461+0000","last_change":"2024-09-13T10:34:33.543507+0000","last_active":"2024-09-13T10:34:35.240461+0000","last_peered":"2024-09-13T10:34:35.240461+0000","last_clean":"2024-09-13T10:34:35.240461+0000","last_became_active":"2024-09-13T10:34:33.543213+0000","last_became_peered":"2024-09-13T10:34:33.543213+0000","last_unstale":"2024-09-13T10:34:35.240461+0000","last_undegraded":"2024-09-13T10:34:35.240461+0000","last_fullsized":"2024-09-13T10:34:35.240461+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:38.464203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.d","version":"98'4","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.278054+0000","last_change":"2024-09-13T10:34:33.543915+0000","last_active":"2024-09-13T10:34:35.278054+0000","last_peered":"2024-09-13T10:34:35.278054+0000","last_clean":"2024-09-13T10:34:35.278054+0000","last_became_active":"2024-09-13T10:34:33.543629+0000","last_became_peered":"2024-09-13T10:34:33.543629+0000","last_unstale":"2024-09-13T10:34:35.278054+0000","last_undegraded":"2024-09-13T10:34:35.278054+0000","last_fullsized":"2024-09-13T10:34:35.278054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:38.711359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.e","version":"99'32","reported_seq":65,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:36.253102+0000","last_change":"2024-09-13T10:34:33.542668+0000","last_active":"2024-09-13T10:34:36.253102+0000","last_peered":"2024-09-13T10:34:36.253102+0000","last_clean":"2024-09-13T10:34:36.253102+0000","last_became_active":"2024-09-13T10:34:33.542370+0000","last_became_peered":"2024-09-13T10:34:33.542370+0000","last_unstale":"2024-09-13T10:34:36.253102+0000","last_undegraded":"2024-09-13T10:34:36.253102+0000","last_fullsized":"2024-09-13T10:34:36.253102+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:52.614236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":138,"num_write":33,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.535468+0000","last_change":"2024-09-13T10:34:32.535468+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.535468+0000","last_undegraded":"2024-09-13T10:34:32.535468+0000","last_fullsized":"2024-09-13T10:34:32.535468+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9],"acting":[8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.535415+0000","last_change":"2024-09-13T10:34:32.535415+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.535415+0000","last_undegraded":"2024-09-13T10:34:32.535415+0000","last_fullsized":"2024-09-13T10:34:32.535415+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1],"acting":[8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540038+0000","last_change":"2024-09-13T10:34:32.540038+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540038+0000","last_undegraded":"2024-09-13T10:34:32.540038+0000","last_fullsized":"2024-09-13T10:34:32.540038+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540560+0000","last_change":"2024-09-13T10:34:32.540560+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540560+0000","last_undegraded":"2024-09-13T10:34:32.540560+0000","last_fullsized":"2024-09-13T10:34:32.540560+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3],"acting":[10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.278027+0000","last_change":"2024-09-13T10:34:33.544036+0000","last_active":"2024-09-13T10:34:35.278027+0000","last_peered":"2024-09-13T10:34:35.278027+0000","last_clean":"2024-09-13T10:34:35.278027+0000","last_became_active":"2024-09-13T10:34:33.543783+0000","last_became_peered":"2024-09-13T10:34:33.543783+0000","last_unstale":"2024-09-13T10:34:35.278027+0000","last_undegraded":"2024-09-13T10:34:35.278027+0000","last_fullsized":"2024-09-13T10:34:35.278027+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:47.732663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540086+0000","last_change":"2024-09-13T10:34:32.540086+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540086+0000","last_undegraded":"2024-09-13T10:34:32.540086+0000","last_fullsized":"2024-09-13T10:34:32.540086+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.277994+0000","last_change":"2024-09-13T10:34:33.544984+0000","last_active":"2024-09-13T10:34:35.277994+0000","last_peered":"2024-09-13T10:34:35.277994+0000","last_clean":"2024-09-13T10:34:35.277994+0000","last_became_active":"2024-09-13T10:34:33.544638+0000","last_became_peered":"2024-09-13T10:34:33.544638+0000","last_unstale":"2024-09-13T10:34:35.277994+0000","last_undegraded":"2024-09-13T10:34:35.277994+0000","last_fullsized":"2024-09-13T10:34:35.277994+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:43.714781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1],"acting":[9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'25","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.266359+0000","last_change":"2024-09-13T10:34:33.544549+0000","last_active":"2024-09-13T10:34:35.266359+0000","last_peered":"2024-09-13T10:34:35.266359+0000","last_clean":"2024-09-13T10:34:35.266359+0000","last_became_active":"2024-09-13T10:34:33.544183+0000","last_became_peered":"2024-09-13T10:34:33.544183+0000","last_unstale":"2024-09-13T10:34:35.266359+0000","last_undegraded":"2024-09-13T10:34:35.266359+0000","last_fullsized":"2024-09-13T10:34:35.266359+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:57.446871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":64,"num_write":17,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540576+0000","last_change":"2024-09-13T10:34:32.540576+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540576+0000","last_undegraded":"2024-09-13T10:34:32.540576+0000","last_fullsized":"2024-09-13T10:34:32.540576+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.278007+0000","last_change":"2024-09-13T10:34:33.543995+0000","last_active":"2024-09-13T10:34:35.278007+0000","last_peered":"2024-09-13T10:34:35.278007+0000","last_clean":"2024-09-13T10:34:35.278007+0000","last_became_active":"2024-09-13T10:34:33.543568+0000","last_became_peered":"2024-09-13T10:34:33.543568+0000","last_unstale":"2024-09-13T10:34:35.278007+0000","last_undegraded":"2024-09-13T10:34:35.278007+0000","last_fullsized":"2024-09-13T10:34:35.278007+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:37.712629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:34.538048+0000","last_change":"2024-09-13T10:34:33.544186+0000","last_active":"2024-09-13T10:34:34.538048+0000","last_peered":"2024-09-13T10:34:34.538048+0000","last_clean":"2024-09-13T10:34:34.538048+0000","last_became_active":"2024-09-13T10:34:33.543949+0000","last_became_peered":"2024-09-13T10:34:33.543949+0000","last_unstale":"2024-09-13T10:34:34.538048+0000","last_undegraded":"2024-09-13T10:34:34.538048+0000","last_fullsized":"2024-09-13T10:34:34.538048+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:33.283173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-09-13T10:34:32.540045+0000","last_change":"2024-09-13T10:34:32.540045+0000","last_active":"2024-09-13T10:34:32.527188+0000","last_peered":"2024-09-13T10:34:32.527188+0000","last_clean":"2024-09-13T10:34:32.527188+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:34:32.540045+0000","last_undegraded":"2024-09-13T10:34:32.540045+0000","last_fullsized":"2024-09-13T10:34:32.540045+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8],"acting":[6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:34:35.240481+0000","last_change":"2024-09-13T10:34:33.546634+0000","last_active":"2024-09-13T10:34:35.240481+0000","last_peered":"2024-09-13T10:34:35.240481+0000","last_clean":"2024-09-13T10:34:35.240481+0000","last_became_active":"2024-09-13T10:34:33.546326+0000","last_became_peered":"2024-09-13T10:34:33.546326+0000","last_unstale":"2024-09-13T10:34:35.240481+0000","last_undegraded":"2024-09-13T10:34:35.240481+0000","last_fullsized":"2024-09-13T10:34:35.240481+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:57.664503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9],"acting":[11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":20,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":240,"num_read_kb":218,"num_write":62,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":40,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":64,"acting":64,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3727360,"data_stored":3702496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1318912,"data_stored":945477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717404,"statfs":{"total":95995035648,"available":95966621696,"internally_reserved":0,"allocated":737280,"data_stored":384893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743452,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":786432,"data_stored":416569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005088,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717392,"statfs":{"total":95995035648,"available":95966609408,"internally_reserved":0,"allocated":749568,"data_stored":392290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28292,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716860,"statfs":{"total":95995035648,"available":95966064640,"internally_reserved":0,"allocated":1294336,"data_stored":929307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717404,"statfs":{"total":95995035648,"available":95966621696,"internally_reserved":0,"allocated":737280,"data_stored":384913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":786432,"data_stored":416569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27776,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717376,"statfs":{"total":95995035648,"available":95966593024,"internally_reserved":0,"allocated":761856,"data_stored":400379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117050,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27788,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717364,"statfs":{"total":95995035648,"available":95966580736,"internally_reserved":0,"allocated":774144,"data_stored":408518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313279,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":786432,"data_stored":416569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":774144,"data_stored":408518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":790528,"data_stored":416589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":20,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":20,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:34:38.607 INFO:tasks.thrashosds.thrasher:Setting 2.8 to [0, 2] 2024-09-13T10:34:38.607 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.8', '0', '2'] 2024-09-13T10:34:38.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 2.8 0 2 2024-09-13T10:34:38.647 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:38.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:38.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:38.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:34:38.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:38.813 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:38.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:38 smithi138 ceph-mon[31033]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:38 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2527547342' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-13T10:34:38.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:38 smithi138 ceph-mon[31033]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T10:34:38.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:38 smithi060 ceph-mon[27375]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:38.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:38 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2527547342' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-13T10:34:38.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:38 smithi060 ceph-mon[27375]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T10:34:38.916 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:38.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:38.943 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:38.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:38 smithi173 ceph-mon[30864]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:38 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2527547342' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-13T10:34:39.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:38 smithi173 ceph-mon[30864]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T10:34:39.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.068 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:39.068 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:39.169 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:39.193 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:39.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.194 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:39.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:39.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:39.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:39.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:39.441 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:39.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T10:34:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:34:39.544 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:39.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.571 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:39.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:39.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.700 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:39.779 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:39.779 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:39.801 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:39.825 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:39.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.826 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:39.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:39 smithi138 ceph-mon[31033]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:39.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:39 smithi138 ceph-mon[31033]: pgmap v313: 33 pgs: 30 active+clean, 3 creating+peering; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 767 KiB/s wr, 81 op/s 2024-09-13T10:34:39.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:39 smithi060 ceph-mon[27375]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:39.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:39 smithi060 ceph-mon[27375]: pgmap v313: 33 pgs: 30 active+clean, 3 creating+peering; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 767 KiB/s wr, 81 op/s 2024-09-13T10:34:39.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:39.949 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:39.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:39.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:39 smithi173 ceph-mon[30864]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:40.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:39 smithi173 ceph-mon[30864]: pgmap v313: 33 pgs: 30 active+clean, 3 creating+peering; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 767 KiB/s wr, 81 op/s 2024-09-13T10:34:40.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:40.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.077 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:40.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:40.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:40.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-13T10:34:40.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:40.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:40.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:40.330 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:40.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:40.433 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:40.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.460 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:40.460 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:40.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:40.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:40.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:40.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:40.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:40.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:40 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:40.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:40 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:40.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:40 smithi060 ceph-mon[27375]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T10:34:40.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:40 smithi060 ceph-mon[27375]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T10:34:40.810 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:40.837 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:40.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:40.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:40 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:40.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:40 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:40.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:40 smithi138 ceph-mon[31033]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T10:34:40.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:40 smithi138 ceph-mon[31033]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T10:34:40.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:40.946 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-13T10:34:40.946 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:34:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:40.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:40.961 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:40.967 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-13T10:34:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:34:40.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-13T10:34:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:34:41.009 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-13T10:34:41.009 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:34:41.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:40 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:41.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:40 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:41.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:40 smithi173 ceph-mon[30864]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T10:34:41.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:40 smithi173 ceph-mon[30864]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T10:34:41.031 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-13T10:34:41.031 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:34:41.052 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-13T10:34:41.053 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:34:41.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:41.074 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-13T10:34:41.074 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:34:41.088 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:41.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.089 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:41.096 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-13T10:34:41.096 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:34:41.117 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8: {} 2024-09-13T10:34:41.118 INFO:teuthology.orchestra.run.smithi060.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:34:41.139 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9: {} 2024-09-13T10:34:41.139 INFO:teuthology.orchestra.run.smithi060.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:34:41.161 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10: {} 2024-09-13T10:34:41.162 INFO:teuthology.orchestra.run.smithi060.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:34:41.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11: {} 2024-09-13T10:34:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:34:41.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:41.216 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:41.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:41.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-13T10:34:41.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:41.354 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:41.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:41.456 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:41.482 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:41.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.484 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:set 2.8 pg_upmap mapping to [0,2] 2024-09-13T10:34:41.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:41.609 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:41.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:41.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:41.737 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:41.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.738 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:41.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:41 smithi060 ceph-mon[27375]: pgmap v315: 33 pgs: 33 active+clean; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 701 KiB/s wr, 104 op/s 2024-09-13T10:34:41.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:41 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4076229267' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:41.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:41 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1348199391' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 2]} : dispatch 2024-09-13T10:34:41.839 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:41.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:41 smithi138 ceph-mon[31033]: pgmap v315: 33 pgs: 33 active+clean; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 701 KiB/s wr, 104 op/s 2024-09-13T10:34:41.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:41 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4076229267' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:41.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:41 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1348199391' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 2]} : dispatch 2024-09-13T10:34:41.863 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:41.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:41.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:41.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:41.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:41 smithi173 ceph-mon[30864]: pgmap v315: 33 pgs: 33 active+clean; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 701 KiB/s wr, 104 op/s 2024-09-13T10:34:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:41 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4076229267' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:42.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:41 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1348199391' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 2]} : dispatch 2024-09-13T10:34:42.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:42.116 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:42.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:42.218 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:42.244 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:42.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.245 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:42.346 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:42.371 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:42.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.373 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:07.871190+0000", 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.51772376999998, 2024-09-13T10:34:42.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054155800000000004, 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871190+0000", 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871190+0000", 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871191+0000", 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.75e-07 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871246+0000", 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5365999999999998e-05 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871275+0000", 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9193e-05 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871282+0000", 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.567e-06 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871341+0000", 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8542000000000003e-05 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871356+0000", 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5054e-05 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871708+0000", 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035275200000000001 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871732+0000", 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3408999999999999e-05 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.400 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:22 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'12, mlcod=16'8)", 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:07.871822+0000", 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.51709231400002, 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00061479500000000001, 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871822+0000", 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871822+0000", 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871822+0000", 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0999999999999998e-07 2024-09-13T10:34:42.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871930+0000", 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107366 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871968+0000", 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8509e-05 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.871976+0000", 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9030000000000004e-06 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.872033+0000", 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7204000000000003e-05 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.872053+0000", 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9437e-05 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.872403+0000", 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035038000000000002 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:07.872436+0000", 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3386000000000002e-05 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.405 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:34 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'20, mlcod=16'18)", 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.147238+0000", 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.24167622200002, 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053272200000000001, 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147238+0000", 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147238+0000", 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147241+0000", 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3960000000000002e-06 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147257+0000", 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591e-05 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147319+0000", 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2581000000000003e-05 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147334+0000", 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4837999999999999e-05 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147415+0000", 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1093000000000004e-05 2024-09-13T10:34:42.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147431+0000", 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5812999999999999e-05 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147741+0000", 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000309652 2024-09-13T10:34:42.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.147771+0000", 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9757999999999999e-05 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:40 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'19)", 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.150098+0000", 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.23881560900003, 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052719400000000003, 2024-09-13T10:34:42.410 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150098+0000", 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150098+0000", 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150099+0000", 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0400000000000005e-07 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150106+0000", 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8870000000000003e-06 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150147+0000", 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0791000000000003e-05 2024-09-13T10:34:42.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150154+0000", 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7340000000000002e-06 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150201+0000", 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7262999999999998e-05 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150220+0000", 2024-09-13T10:34:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9477999999999999e-05 2024-09-13T10:34:42.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150597+0000", 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037685600000000001 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150626+0000", 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8381000000000002e-05 2024-09-13T10:34:42.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:41 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'23, mlcod=16'19)", 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.150257+0000", 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.23865695299997, 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067215899999999997, 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150257+0000", 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150257+0000", 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150258+0000", 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4199999999999995e-07 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150309+0000", 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1498000000000001e-05 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150338+0000", 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8920000000000001e-05 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150344+0000", 2024-09-13T10:34:42.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8189999999999997e-06 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150410+0000", 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5988999999999994e-05 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150431+0000", 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1517000000000001e-05 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150892+0000", 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046036299999999998 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.150929+0000", 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7410999999999998e-05 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:50 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'28)", 2024-09-13T10:34:42.421 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.154763+0000", 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.23415114599999, 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051660499999999995, 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154763+0000", 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154763+0000", 2024-09-13T10:34:42.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154764+0000", 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1699999999999997e-07 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154808+0000", 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4733e-05 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154843+0000", 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5151000000000003e-05 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154851+0000", 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7130000000000005e-06 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154894+0000", 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2923000000000002e-05 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.154914+0000", 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9459000000000001e-05 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.155245+0000", 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033119699999999998 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.155279+0000", 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4712000000000002e-05 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:53 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.156659+0000", 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.23225537500002, 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00071416700000000004, 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.426 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156659+0000", 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156659+0000", 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156659+0000", 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9999999999999997e-07 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156714+0000", 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4693e-05 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156751+0000", 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6838000000000002e-05 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156757+0000", 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6159999999999998e-06 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156803+0000", 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5763000000000002e-05 2024-09-13T10:34:42.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.156816+0000", 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3247999999999999e-05 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.157345+0000", 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052856500000000005 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.157373+0000", 2024-09-13T10:34:42.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:66 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'40)", 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.305251+0000", 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.08366262300001, 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00078293600000000003, 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305251+0000", 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305251+0000", 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305252+0000", 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6499999999999999e-07 2024-09-13T10:34:42.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305273+0000", 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1284000000000002e-05 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305326+0000", 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2509999999999997e-05 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305332+0000", 2024-09-13T10:34:42.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7519999999999999e-06 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305495+0000", 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000162583 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305517+0000", 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1749000000000001e-05 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.305984+0000", 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046773000000000002 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.306034+0000", 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9863e-05 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:73 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'51, mlcod=16'48)", 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.309170+0000", 2024-09-13T10:34:42.435 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.07974427699997, 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00056681699999999995, 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309170+0000", 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309170+0000", 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309170+0000", 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3300000000000002e-07 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309174+0000", 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8099999999999999e-06 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309201+0000", 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6937999999999999e-05 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309207+0000", 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6559999999999997e-06 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309245+0000", 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8655000000000002e-05 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309265+0000", 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0089999999999999e-05 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309702+0000", 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000436787 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309737+0000", 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4347999999999997e-05 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.439 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:75 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'49)", 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.309963+0000", 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.07895147099998, 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053280799999999996, 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309963+0000", 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309963+0000", 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309963+0000", 2024-09-13T10:34:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3499999999999996e-07 2024-09-13T10:34:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.309986+0000", 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2699000000000001e-05 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.310015+0000", 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8748999999999999e-05 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.310020+0000", 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4389999999999999e-06 2024-09-13T10:34:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.310060+0000", 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9792000000000001e-05 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.310072+0000", 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2086000000000001e-05 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.310453+0000", 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00038148799999999998 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.310495+0000", 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2020000000000001e-05 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:79 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'57, mlcod=16'53)", 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.312276+0000", 2024-09-13T10:34:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.07663814699998, 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051954699999999998, 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312276+0000", 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312276+0000", 2024-09-13T10:34:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312276+0000", 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7899999999999998e-07 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312308+0000", 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1687999999999999e-05 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312345+0000", 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6913999999999999e-05 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312353+0000", 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8790000000000003e-06 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312398+0000", 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4950000000000002e-05 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312417+0000", 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8838000000000002e-05 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312767+0000", 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035053599999999998 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.312795+0000", 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8163000000000001e-05 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:90 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'64, mlcod=16'60)", 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.370470+0000", 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.01844428800001, 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00066041600000000004, 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370470+0000", 2024-09-13T10:34:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370470+0000", 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370470+0000", 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9299999999999999e-07 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370493+0000", 2024-09-13T10:34:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.27e-05 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370514+0000", 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1480000000000001e-05 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370518+0000", 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.681e-06 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370646+0000", 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128085 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.370666+0000", 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.984e-05 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.371077+0000", 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041143499999999998 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.371130+0000", 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2802e-05 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:96 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'66)", 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.373647+0000", 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.01526662999999, 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053069399999999996, 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373647+0000", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373647+0000", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373648+0000", 2024-09-13T10:34:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2500000000000005e-07 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373679+0000", 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0991999999999997e-05 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373718+0000", 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9425999999999998e-05 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373725+0000", 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.562e-06 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373755+0000", 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9867000000000001e-05 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.373765+0000", 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0475000000000001e-05 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.374152+0000", 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00038630199999999999 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.374178+0000", 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6444999999999998e-05 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:99 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'69)", 2024-09-13T10:34:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.375213+0000", 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.01370113899998, 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054251899999999999, 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375213+0000", 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375213+0000", 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375214+0000", 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5400000000000001e-07 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375236+0000", 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.23e-05 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375285+0000", 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8773999999999998e-05 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375296+0000", 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1739999999999999e-05 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375336+0000", 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9805999999999998e-05 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375348+0000", 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1596999999999999e-05 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375725+0000", 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037745900000000002 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.375755+0000", 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0188999999999998e-05 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:102 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'76, mlcod=16'73)", 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.377167+0000", 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.01174690400001, 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069891699999999996, 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377167+0000", 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377167+0000", 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377168+0000", 2024-09-13T10:34:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0399999999999996e-07 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377194+0000", 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6353e-05 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377234+0000", 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0123000000000002e-05 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377243+0000", 2024-09-13T10:34:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8699999999999998e-06 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377298+0000", 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4960999999999998e-05 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377319+0000", 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0686000000000002e-05 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377827+0000", 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000508469 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377866+0000", 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8850999999999998e-05 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:103 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.377519+0000", 2024-09-13T10:34:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.01139506700002, 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051397300000000003, 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377519+0000", 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377519+0000", 2024-09-13T10:34:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377520+0000", 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377575+0000", 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5025999999999998e-05 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377612+0000", 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7403000000000001e-05 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377618+0000", 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2639999999999997e-06 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377666+0000", 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8086e-05 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377686+0000", 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9755999999999999e-05 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377970+0000", 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028394800000000001 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378033+0000", 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2817000000000005e-05 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:104 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'78, mlcod=16'73)", 2024-09-13T10:34:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.377986+0000", 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.01092821899999, 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051380800000000004, 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377986+0000", 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377986+0000", 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.377986+0000", 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4099999999999999e-07 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378004+0000", 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7810000000000001e-05 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378035+0000", 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1245000000000003e-05 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378040+0000", 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9089999999999999e-06 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378078+0000", 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8104e-05 2024-09-13T10:34:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378098+0000", 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9531e-05 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378467+0000", 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036900700000000003 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.378500+0000", 2024-09-13T10:34:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2761e-05 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:112 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'82, mlcod=16'80)", 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.386874+0000", 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.002039803, 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054396499999999999, 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386874+0000", 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386874+0000", 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386875+0000", 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.06e-07 2024-09-13T10:34:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386880+0000", 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.809e-06 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386906+0000", 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6035999999999999e-05 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386912+0000", 2024-09-13T10:34:42.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386952+0000", 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9209999999999999e-05 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.386964+0000", 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2002e-05 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.387385+0000", 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042093399999999998 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.387418+0000", 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3584999999999998e-05 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.14156.0:117 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'87, mlcod=16'83)", 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:29:08.388817+0000", 2024-09-13T10:34:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 334.000097122, 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000544565, 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388817+0000", 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388817+0000", 2024-09-13T10:34:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388818+0000", 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6700000000000003e-07 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388845+0000", 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7589999999999998e-05 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388881+0000", 2024-09-13T10:34:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5601e-05 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388887+0000", 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388931+0000", 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3535000000000002e-05 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:34:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.388952+0000", 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1335000000000002e-05 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.389319+0000", 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036664999999999999 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:29:08.389361+0000", 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2896999999999999e-05 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPull(1.0 e40/39 cost 1049576)", 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:30:50.648105+0000", 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 231.740808792, 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053683699999999997, 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648105+0000", 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648105+0000", 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648106+0000", 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9900000000000009e-07 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648123+0000", 2024-09-13T10:34:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6388e-05 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648163+0000", 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0652999999999999e-05 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648177+0000", 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3465999999999999e-05 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:34:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648232+0000", 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5164999999999998e-05 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:30:50.648642+0000", 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000410166 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:34:42.489 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:34:42.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:42.512 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:42.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:42.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:42.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.640 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:42.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:42.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:42.750 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1348199391' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 2]}]': finished 2024-09-13T10:34:42.750 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:42 smithi060 ceph-mon[27375]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T10:34:42.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.767 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:42.767 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:42 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1348199391' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 2]}]': finished 2024-09-13T10:34:42.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:42 smithi138 ceph-mon[31033]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T10:34:42.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:42.894 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:42.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:42.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:42.996 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:43.019 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:43.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.020 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:43.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:42 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1348199391' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 2]}]': finished 2024-09-13T10:34:43.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:42 smithi173 ceph-mon[30864]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T10:34:43.121 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:43.147 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:43.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:43.249 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:43.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.272 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:43.272 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:43.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-13T10:34:43.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:43.399 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:43.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:43.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:43.526 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:43.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:43.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:43.652 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:43.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:43.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:43.778 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:43.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.779 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:43.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:34:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:43 smithi138 ceph-mon[31033]: pgmap v318: 33 pgs: 33 active+clean; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 366 op/s 2024-09-13T10:34:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:43 smithi138 ceph-mon[31033]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T10:34:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:43 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:43.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:43 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:43.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:43 smithi060 ceph-mon[27375]: pgmap v318: 33 pgs: 33 active+clean; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 366 op/s 2024-09-13T10:34:43.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:43 smithi060 ceph-mon[27375]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T10:34:43.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:43 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:43.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:43 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:43.880 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:43.902 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:43.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:43.903 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:43.920 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:44.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:44.026 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:43 smithi173 ceph-mon[30864]: pgmap v318: 33 pgs: 33 active+clean; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 366 op/s 2024-09-13T10:34:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:43 smithi173 ceph-mon[30864]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T10:34:44.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:43 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:44.028 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:43 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:44.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:44.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:44.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:44.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:44.278 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:44.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:44.380 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:44.406 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:44.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:44.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:44.535 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:44.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:44.635 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:34:44.635 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T10:34:44.635 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T10:34:44.635 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:34:44.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:44.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.662 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:44.663 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:44.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:44.774 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:44 smithi060 ceph-mon[27375]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T10:34:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:44.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:44.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:44 smithi138 ceph-mon[31033]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T10:34:44.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:44.915 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:44.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:44.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:45.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:45.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:44 smithi173 ceph-mon[30864]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T10:34:45.039 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:45.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-13T10:34:45.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:45.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:45.266 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:45.290 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:45.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.291 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:45.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:45.419 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:45.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:45.521 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:45.543 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:45.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.544 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:45.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:45.669 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:45.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:45.713 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:45 smithi060 ceph-mon[27375]: pgmap v321: 33 pgs: 1 peering, 32 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 107 KiB/s wr, 326 op/s 2024-09-13T10:34:45.713 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:45 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/634384033' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:34:45.713 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:45 smithi060 ceph-mon[27375]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:45.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:45 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/634384033' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-13T10:34:45.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:45 smithi060 ceph-mon[27375]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T10:34:45.729 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:45.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-13T10:34:45.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:45.820 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:45.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:45 smithi138 ceph-mon[31033]: pgmap v321: 33 pgs: 1 peering, 32 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 107 KiB/s wr, 326 op/s 2024-09-13T10:34:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:45 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/634384033' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:34:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:45 smithi138 ceph-mon[31033]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:45 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/634384033' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-13T10:34:45.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:45 smithi138 ceph-mon[31033]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T10:34:45.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:45.948 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:45.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:45.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:45 smithi173 ceph-mon[30864]: pgmap v321: 33 pgs: 1 peering, 32 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 107 KiB/s wr, 326 op/s 2024-09-13T10:34:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:45 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/634384033' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:34:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:45 smithi173 ceph-mon[30864]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:45 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/634384033' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-13T10:34:46.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:45 smithi173 ceph-mon[30864]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T10:34:46.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:46.071 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:46.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:46.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:46.196 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:46.197 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:46.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:46.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:46.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:46.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:46.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:46.445 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:46.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.446 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:46.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:46.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:46.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:46.699 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:46.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:46.801 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:46.828 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:46.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.829 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:46.882 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-13T10:34:46.882 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:34:46.904 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-13T10:34:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:34:46.925 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-13T10:34:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:34:46.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:46.946 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-13T10:34:46.946 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:34:46.955 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:46.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:46.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:46.969 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-13T10:34:46.969 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:34:46.990 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-13T10:34:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:34:47.012 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-13T10:34:47.012 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:34:47.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:47.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:47.085 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T10:34:47.085 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:34:47.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:47.208 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:47.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.209 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:47.235 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-13T10:34:47.353 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:34:47.353 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8: {} 2024-09-13T10:34:47.354 INFO:teuthology.orchestra.run.smithi060.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:34:47.354 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9: {} 2024-09-13T10:34:47.354 INFO:teuthology.orchestra.run.smithi060.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:34:47.354 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10: {} 2024-09-13T10:34:47.355 INFO:teuthology.orchestra.run.smithi060.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:34:47.355 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11: {} 2024-09-13T10:34:47.355 INFO:teuthology.orchestra.run.smithi060.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:34:47.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:47.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:47.383 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:47.484 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:47.510 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:47.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.511 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:47.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:47.635 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:47.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:47.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:47.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.764 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:47.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:47 smithi138 ceph-mon[31033]: pgmap v323: 33 pgs: 1 peering, 32 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 158 KiB/s wr, 370 op/s 2024-09-13T10:34:47.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:47 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1125236067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:47.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:47 smithi060 ceph-mon[27375]: pgmap v323: 33 pgs: 1 peering, 32 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 158 KiB/s wr, 370 op/s 2024-09-13T10:34:47.872 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:47 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1125236067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:47.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:47.890 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:47.890 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:47.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-13T10:34:47.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:48.014 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:48.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:48.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:47 smithi173 ceph-mon[30864]: pgmap v323: 33 pgs: 1 peering, 32 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 158 KiB/s wr, 370 op/s 2024-09-13T10:34:48.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:47 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1125236067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:48.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:48.139 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:48.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.140 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:48.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-13T10:34:48.241 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:48.267 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:48.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.268 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:48.369 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:48.391 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:48.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:48.493 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:48.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.519 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:48.519 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:48.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:48.642 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:48.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:48.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:48.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:48.821 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-13T10:34:48.822 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:34:48.822 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-13T10:34:48.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:34:48.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:48.895 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:48.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:48.896 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:48.944 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:48.978 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:48.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:49.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:49.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:49.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:49.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:49.145 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:49.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:49.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.273 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:49.274 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:49.375 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:49.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.400 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:49.400 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:49.504 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:49.531 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:49.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:49.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:49.656 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:49.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.657 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:49.658 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T10:34:49.658 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:34:49.758 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:49.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:49.873 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:49 smithi060 ceph-mon[27375]: pgmap v324: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 116 KiB/s wr, 272 op/s; 6 B/s, 4 keys/s, 0 objects/s recovering 2024-09-13T10:34:49.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:49.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:49.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:49.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:50.011 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:50.027 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:49 smithi173 ceph-mon[30864]: pgmap v324: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 116 KiB/s wr, 272 op/s; 6 B/s, 4 keys/s, 0 objects/s recovering 2024-09-13T10:34:50.037 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:50.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.038 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:50.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:49 smithi138 ceph-mon[31033]: pgmap v324: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 116 KiB/s wr, 272 op/s; 6 B/s, 4 keys/s, 0 objects/s recovering 2024-09-13T10:34:50.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:50.165 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:50.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:50.267 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:50.293 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:50.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.294 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:50.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:50.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-13T10:34:50.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.424 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:50.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:50.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:50.550 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:50.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:50.652 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:50.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:50.780 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:50.806 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:50.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:50.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:34:50.909 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:50.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:50.936 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:50.936 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:51.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:51.064 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:51.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.065 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:51.066 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-13T10:34:51.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:51.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.192 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:51.192 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:51.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:51.302 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:51.319 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:51.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:34:51.323 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:51 smithi060 ceph-mon[27375]: pgmap v325: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 94 KiB/s wr, 220 op/s; 5 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:51.323 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:51 smithi060 ceph-mon[27375]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T10:34:51.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:51 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:51.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:51 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:51.324 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:51 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1369561533' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-13T10:34:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:34:51.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:51 smithi138 ceph-mon[31033]: pgmap v325: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 94 KiB/s wr, 220 op/s; 5 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:51.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:51 smithi138 ceph-mon[31033]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T10:34:51.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:51 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:51.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:51 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:51.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:51 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1369561533' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-13T10:34:51.421 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:51.447 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:51.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:51.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:51 smithi173 ceph-mon[30864]: pgmap v325: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 94 KiB/s wr, 220 op/s; 5 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:51.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:51 smithi173 ceph-mon[30864]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T10:34:51.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:51 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:51.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:51 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:51.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:51 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1369561533' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-13T10:34:51.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:51.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:51.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:51.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.698 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:51.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:51.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:51.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:51.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:51.924 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:51.951 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:51.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:51.952 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:52.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:52.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.081 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:52.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:52.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:52 smithi060 ceph-mon[27375]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:52 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1369561533' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-13T10:34:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:52 smithi060 ceph-mon[27375]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T10:34:52.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:52 smithi060 ceph-mon[27375]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:52.182 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:52.207 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:52.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.208 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:52.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:52.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.334 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:52.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:52.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:52 smithi138 ceph-mon[31033]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:52.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:52 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1369561533' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-13T10:34:52.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:52 smithi138 ceph-mon[31033]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T10:34:52.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:52 smithi138 ceph-mon[31033]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:52.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:52.481 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:52.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.483 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:52 smithi173 ceph-mon[30864]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:34:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:52 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1369561533' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-13T10:34:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:52 smithi173 ceph-mon[30864]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T10:34:52.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:52 smithi173 ceph-mon[30864]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:34:52.549 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-13T10:34:51.066120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590742,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":3274,"num_read_kb":11351,"num_write":2063,"num_write_kb":8158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1586822,"num_keys_recovered":26,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1495,"ondisk_log_size":1495,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352332,"kb_used_data":11332,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589492,"statfs":{"total":1151940427776,"available":1151579639808,"internally_reserved":0,"allocated":11603968,"data_stored":6715892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1830,"num_read_kb":1830,"num_write":1373,"num_write_kb":1373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":38,"num_keys_recovered":26,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.086937"},"pg_stats":[{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266157+0000","last_change":"2024-09-13T10:34:33.543301+0000","last_active":"2024-09-13T10:34:45.266157+0000","last_peered":"2024-09-13T10:34:45.266157+0000","last_clean":"2024-09-13T10:34:45.266157+0000","last_became_active":"2024-09-13T10:34:33.543049+0000","last_became_peered":"2024-09-13T10:34:33.543049+0000","last_unstale":"2024-09-13T10:34:45.266157+0000","last_undegraded":"2024-09-13T10:34:45.266157+0000","last_fullsized":"2024-09-13T10:34:45.266157+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:59.943935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"98'7","reported_seq":35,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.230648+0000","last_change":"2024-09-13T10:34:33.545736+0000","last_active":"2024-09-13T10:34:45.230648+0000","last_peered":"2024-09-13T10:34:45.230648+0000","last_clean":"2024-09-13T10:34:45.230648+0000","last_became_active":"2024-09-13T10:34:33.545495+0000","last_became_peered":"2024-09-13T10:34:33.545495+0000","last_unstale":"2024-09-13T10:34:45.230648+0000","last_undegraded":"2024-09-13T10:34:45.230648+0000","last_fullsized":"2024-09-13T10:34:45.230648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:38.470069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1d","version":"99'3","reported_seq":32,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266352+0000","last_change":"2024-09-13T10:34:33.554313+0000","last_active":"2024-09-13T10:34:45.266352+0000","last_peered":"2024-09-13T10:34:45.266352+0000","last_clean":"2024-09-13T10:34:45.266352+0000","last_became_active":"2024-09-13T10:34:33.554007+0000","last_became_peered":"2024-09-13T10:34:33.554007+0000","last_unstale":"2024-09-13T10:34:45.266352+0000","last_undegraded":"2024-09-13T10:34:45.266352+0000","last_fullsized":"2024-09-13T10:34:45.266352+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:47.338552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7],"acting":[10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266339+0000","last_change":"2024-09-13T10:34:33.554309+0000","last_active":"2024-09-13T10:34:45.266339+0000","last_peered":"2024-09-13T10:34:45.266339+0000","last_clean":"2024-09-13T10:34:45.266339+0000","last_became_active":"2024-09-13T10:34:33.553995+0000","last_became_peered":"2024-09-13T10:34:33.553995+0000","last_unstale":"2024-09-13T10:34:45.266339+0000","last_undegraded":"2024-09-13T10:34:45.266339+0000","last_fullsized":"2024-09-13T10:34:45.266339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:50.612089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7],"acting":[10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.b","version":"99'2","reported_seq":29,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.058960+0000","last_change":"2024-09-13T10:34:33.544342+0000","last_active":"2024-09-13T10:34:45.058960+0000","last_peered":"2024-09-13T10:34:45.058960+0000","last_clean":"2024-09-13T10:34:45.058960+0000","last_became_active":"2024-09-13T10:34:33.542737+0000","last_became_peered":"2024-09-13T10:34:33.542737+0000","last_unstale":"2024-09-13T10:34:45.058960+0000","last_undegraded":"2024-09-13T10:34:45.058960+0000","last_fullsized":"2024-09-13T10:34:45.058960+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:46.476464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.a","version":"99'2","reported_seq":28,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.731310+0000","last_change":"2024-09-13T10:34:33.544265+0000","last_active":"2024-09-13T10:34:45.731310+0000","last_peered":"2024-09-13T10:34:45.731310+0000","last_clean":"2024-09-13T10:34:45.731310+0000","last_became_active":"2024-09-13T10:34:33.544016+0000","last_became_peered":"2024-09-13T10:34:33.544016+0000","last_unstale":"2024-09-13T10:34:45.731310+0000","last_undegraded":"2024-09-13T10:34:45.731310+0000","last_fullsized":"2024-09-13T10:34:45.731310+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:32.957770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3],"acting":[11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"100'5","reported_seq":60,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-13T10:34:50.057002+0000","last_change":"2024-09-13T10:34:33.545215+0000","last_active":"2024-09-13T10:34:50.057002+0000","last_peered":"2024-09-13T10:34:50.057002+0000","last_clean":"2024-09-13T10:34:50.057002+0000","last_became_active":"2024-09-13T10:34:33.544543+0000","last_became_peered":"2024-09-13T10:34:33.544543+0000","last_unstale":"2024-09-13T10:34:50.057002+0000","last_undegraded":"2024-09-13T10:34:50.057002+0000","last_fullsized":"2024-09-13T10:34:50.057002+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:01.397026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":48,"num_write":5,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"100'17","reported_seq":33,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266041+0000","last_change":"2024-09-13T10:34:42.827160+0000","last_active":"2024-09-13T10:34:45.266041+0000","last_peered":"2024-09-13T10:34:45.266041+0000","last_clean":"2024-09-13T10:34:45.266041+0000","last_became_active":"2024-09-13T10:34:42.810248+0000","last_became_peered":"2024-09-13T10:34:42.810248+0000","last_unstale":"2024-09-13T10:34:45.266041+0000","last_undegraded":"2024-09-13T10:34:45.266041+0000","last_fullsized":"2024-09-13T10:34:45.266041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:01.794210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":26,"num_write":14,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":38,"num_keys_recovered":26,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"98'23","reported_seq":71,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.056975+0000","last_change":"2024-09-13T10:34:33.547237+0000","last_active":"2024-09-13T10:34:45.056975+0000","last_peered":"2024-09-13T10:34:45.056975+0000","last_clean":"2024-09-13T10:34:45.056975+0000","last_became_active":"2024-09-13T10:34:33.546968+0000","last_became_peered":"2024-09-13T10:34:33.546968+0000","last_unstale":"2024-09-13T10:34:45.056975+0000","last_undegraded":"2024-09-13T10:34:45.056975+0000","last_fullsized":"2024-09-13T10:34:45.056975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:36.648197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":40,"num_read_kb":39,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"106'1173","reported_seq":1242,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-13T10:34:50.057022+0000","last_change":"2024-09-13T10:34:33.540613+0000","last_active":"2024-09-13T10:34:50.057022+0000","last_peered":"2024-09-13T10:34:50.057022+0000","last_clean":"2024-09-13T10:34:50.057022+0000","last_became_active":"2024-09-13T10:34:33.539414+0000","last_became_peered":"2024-09-13T10:34:33.539414+0000","last_unstale":"2024-09-13T10:34:50.057022+0000","last_undegraded":"2024-09-13T10:34:50.057022+0000","last_fullsized":"2024-09-13T10:34:50.057022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":1173,"log_dups_size":0,"ondisk_log_size":1173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:32.772743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2507,"num_read_kb":2474,"num_write":1758,"num_write_kb":1731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.058941+0000","last_change":"2024-09-13T10:34:33.545790+0000","last_active":"2024-09-13T10:34:45.058941+0000","last_peered":"2024-09-13T10:34:45.058941+0000","last_clean":"2024-09-13T10:34:45.058941+0000","last_became_active":"2024-09-13T10:34:33.545516+0000","last_became_peered":"2024-09-13T10:34:33.545516+0000","last_unstale":"2024-09-13T10:34:45.058941+0000","last_undegraded":"2024-09-13T10:34:45.058941+0000","last_fullsized":"2024-09-13T10:34:45.058941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:34.838708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-13T10:34:50.057022+0000","last_change":"2024-09-13T10:34:33.544792+0000","last_active":"2024-09-13T10:34:50.057022+0000","last_peered":"2024-09-13T10:34:50.057022+0000","last_clean":"2024-09-13T10:34:50.057022+0000","last_became_active":"2024-09-13T10:34:33.544466+0000","last_became_peered":"2024-09-13T10:34:33.544466+0000","last_unstale":"2024-09-13T10:34:50.057022+0000","last_undegraded":"2024-09-13T10:34:50.057022+0000","last_fullsized":"2024-09-13T10:34:50.057022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:51.834557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":31,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-13T10:34:50.393527+0000","last_change":"2024-09-13T10:34:33.549230+0000","last_active":"2024-09-13T10:34:50.393527+0000","last_peered":"2024-09-13T10:34:50.393527+0000","last_clean":"2024-09-13T10:34:50.393527+0000","last_became_active":"2024-09-13T10:34:33.548580+0000","last_became_peered":"2024-09-13T10:34:33.548580+0000","last_unstale":"2024-09-13T10:34:50.393527+0000","last_undegraded":"2024-09-13T10:34:50.393527+0000","last_fullsized":"2024-09-13T10:34:50.393527+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:59.968990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'15","reported_seq":42,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266162+0000","last_change":"2024-09-13T10:34:33.543767+0000","last_active":"2024-09-13T10:34:45.266162+0000","last_peered":"2024-09-13T10:34:45.266162+0000","last_clean":"2024-09-13T10:34:45.266162+0000","last_became_active":"2024-09-13T10:34:33.543529+0000","last_became_peered":"2024-09-13T10:34:33.543529+0000","last_unstale":"2024-09-13T10:34:45.266162+0000","last_undegraded":"2024-09-13T10:34:45.266162+0000","last_fullsized":"2024-09-13T10:34:45.266162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:52.434490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"99'20","reported_seq":65,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266337+0000","last_change":"2024-09-13T10:34:33.543759+0000","last_active":"2024-09-13T10:34:45.266337+0000","last_peered":"2024-09-13T10:34:45.266337+0000","last_clean":"2024-09-13T10:34:45.266337+0000","last_became_active":"2024-09-13T10:34:33.543288+0000","last_became_peered":"2024-09-13T10:34:33.543288+0000","last_unstale":"2024-09-13T10:34:45.266337+0000","last_undegraded":"2024-09-13T10:34:45.266337+0000","last_fullsized":"2024-09-13T10:34:45.266337+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:55.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":110,"num_read_kb":86,"num_write":19,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-13T10:34:50.393531+0000","last_change":"2024-09-13T10:34:33.546377+0000","last_active":"2024-09-13T10:34:50.393531+0000","last_peered":"2024-09-13T10:34:50.393531+0000","last_clean":"2024-09-13T10:34:50.393531+0000","last_became_active":"2024-09-13T10:34:33.546143+0000","last_became_peered":"2024-09-13T10:34:33.546143+0000","last_unstale":"2024-09-13T10:34:50.393531+0000","last_undegraded":"2024-09-13T10:34:50.393531+0000","last_fullsized":"2024-09-13T10:34:50.393531+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:43.182551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'92","reported_seq":163,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266462+0000","last_change":"2024-09-13T10:34:13.003713+0000","last_active":"2024-09-13T10:34:45.266462+0000","last_peered":"2024-09-13T10:34:45.266462+0000","last_clean":"2024-09-13T10:34:45.266462+0000","last_became_active":"2024-09-13T10:33:33.765804+0000","last_became_peered":"2024-09-13T10:33:33.765804+0000","last_unstale":"2024-09-13T10:34:45.266462+0000","last_undegraded":"2024-09-13T10:34:45.266462+0000","last_fullsized":"2024-09-13T10:34:45.266462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:16.331942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.731310+0000","last_change":"2024-09-13T10:34:33.543507+0000","last_active":"2024-09-13T10:34:45.731310+0000","last_peered":"2024-09-13T10:34:45.731310+0000","last_clean":"2024-09-13T10:34:45.731310+0000","last_became_active":"2024-09-13T10:34:33.543213+0000","last_became_peered":"2024-09-13T10:34:33.543213+0000","last_unstale":"2024-09-13T10:34:45.731310+0000","last_undegraded":"2024-09-13T10:34:45.731310+0000","last_fullsized":"2024-09-13T10:34:45.731310+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:38.464203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.d","version":"100'25","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266513+0000","last_change":"2024-09-13T10:34:33.543915+0000","last_active":"2024-09-13T10:34:45.266513+0000","last_peered":"2024-09-13T10:34:45.266513+0000","last_clean":"2024-09-13T10:34:45.266513+0000","last_became_active":"2024-09-13T10:34:33.543629+0000","last_became_peered":"2024-09-13T10:34:33.543629+0000","last_unstale":"2024-09-13T10:34:45.266513+0000","last_undegraded":"2024-09-13T10:34:45.266513+0000","last_fullsized":"2024-09-13T10:34:45.266513+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:38.711359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":124,"num_read_kb":109,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.e","version":"100'42","reported_seq":94,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266208+0000","last_change":"2024-09-13T10:34:33.542668+0000","last_active":"2024-09-13T10:34:45.266208+0000","last_peered":"2024-09-13T10:34:45.266208+0000","last_clean":"2024-09-13T10:34:45.266208+0000","last_became_active":"2024-09-13T10:34:33.542370+0000","last_became_peered":"2024-09-13T10:34:33.542370+0000","last_unstale":"2024-09-13T10:34:45.266208+0000","last_undegraded":"2024-09-13T10:34:45.266208+0000","last_fullsized":"2024-09-13T10:34:45.266208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:52.614236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":171,"num_write":44,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"100'11","reported_seq":44,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.058975+0000","last_change":"2024-09-13T10:34:33.545023+0000","last_active":"2024-09-13T10:34:45.058975+0000","last_peered":"2024-09-13T10:34:45.058975+0000","last_clean":"2024-09-13T10:34:45.058975+0000","last_became_active":"2024-09-13T10:34:33.544741+0000","last_became_peered":"2024-09-13T10:34:33.544741+0000","last_unstale":"2024-09-13T10:34:45.058975+0000","last_undegraded":"2024-09-13T10:34:45.058975+0000","last_fullsized":"2024-09-13T10:34:45.058975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:55.838450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":8204,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9],"acting":[8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.10","version":"100'2","reported_seq":29,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.058898+0000","last_change":"2024-09-13T10:34:33.539175+0000","last_active":"2024-09-13T10:34:45.058898+0000","last_peered":"2024-09-13T10:34:45.058898+0000","last_clean":"2024-09-13T10:34:45.058898+0000","last_became_active":"2024-09-13T10:34:33.538912+0000","last_became_peered":"2024-09-13T10:34:33.538912+0000","last_unstale":"2024-09-13T10:34:45.058898+0000","last_undegraded":"2024-09-13T10:34:45.058898+0000","last_fullsized":"2024-09-13T10:34:45.058898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:33.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1],"acting":[8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"100'10","reported_seq":41,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.056826+0000","last_change":"2024-09-13T10:34:33.544408+0000","last_active":"2024-09-13T10:34:45.056826+0000","last_peered":"2024-09-13T10:34:45.056826+0000","last_clean":"2024-09-13T10:34:45.056826+0000","last_became_active":"2024-09-13T10:34:33.544090+0000","last_became_peered":"2024-09-13T10:34:33.544090+0000","last_unstale":"2024-09-13T10:34:45.056826+0000","last_undegraded":"2024-09-13T10:34:45.056826+0000","last_fullsized":"2024-09-13T10:34:45.056826+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:52.271972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"98'1","reported_seq":29,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266328+0000","last_change":"2024-09-13T10:34:33.545516+0000","last_active":"2024-09-13T10:34:45.266328+0000","last_peered":"2024-09-13T10:34:45.266328+0000","last_clean":"2024-09-13T10:34:45.266328+0000","last_became_active":"2024-09-13T10:34:33.545226+0000","last_became_peered":"2024-09-13T10:34:33.545226+0000","last_unstale":"2024-09-13T10:34:45.266328+0000","last_undegraded":"2024-09-13T10:34:45.266328+0000","last_fullsized":"2024-09-13T10:34:45.266328+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:40.301967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3],"acting":[10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266418+0000","last_change":"2024-09-13T10:34:33.544036+0000","last_active":"2024-09-13T10:34:45.266418+0000","last_peered":"2024-09-13T10:34:45.266418+0000","last_clean":"2024-09-13T10:34:45.266418+0000","last_became_active":"2024-09-13T10:34:33.543783+0000","last_became_peered":"2024-09-13T10:34:33.543783+0000","last_unstale":"2024-09-13T10:34:45.266418+0000","last_undegraded":"2024-09-13T10:34:45.266418+0000","last_fullsized":"2024-09-13T10:34:45.266418+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:47.732663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'3","reported_seq":31,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.056847+0000","last_change":"2024-09-13T10:34:33.551111+0000","last_active":"2024-09-13T10:34:45.056847+0000","last_peered":"2024-09-13T10:34:45.056847+0000","last_clean":"2024-09-13T10:34:45.056847+0000","last_became_active":"2024-09-13T10:34:33.550751+0000","last_became_peered":"2024-09-13T10:34:33.550751+0000","last_unstale":"2024-09-13T10:34:45.056847+0000","last_undegraded":"2024-09-13T10:34:45.056847+0000","last_fullsized":"2024-09-13T10:34:45.056847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:46.490729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.15","version":"100'4","reported_seq":34,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266438+0000","last_change":"2024-09-13T10:34:33.544984+0000","last_active":"2024-09-13T10:34:45.266438+0000","last_peered":"2024-09-13T10:34:45.266438+0000","last_clean":"2024-09-13T10:34:45.266438+0000","last_became_active":"2024-09-13T10:34:33.544638+0000","last_became_peered":"2024-09-13T10:34:33.544638+0000","last_unstale":"2024-09-13T10:34:45.266438+0000","last_undegraded":"2024-09-13T10:34:45.266438+0000","last_fullsized":"2024-09-13T10:34:45.266438+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:43.714781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":5,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1],"acting":[9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"99'25","reported_seq":65,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-13T10:34:50.393541+0000","last_change":"2024-09-13T10:34:33.544549+0000","last_active":"2024-09-13T10:34:50.393541+0000","last_peered":"2024-09-13T10:34:50.393541+0000","last_clean":"2024-09-13T10:34:50.393541+0000","last_became_active":"2024-09-13T10:34:33.544183+0000","last_became_peered":"2024-09-13T10:34:33.544183+0000","last_unstale":"2024-09-13T10:34:50.393541+0000","last_undegraded":"2024-09-13T10:34:50.393541+0000","last_fullsized":"2024-09-13T10:34:50.393541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:57.446871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":64,"num_write":17,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"99'7","reported_seq":36,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.056923+0000","last_change":"2024-09-13T10:34:33.547270+0000","last_active":"2024-09-13T10:34:45.056923+0000","last_peered":"2024-09-13T10:34:45.056923+0000","last_clean":"2024-09-13T10:34:45.056923+0000","last_became_active":"2024-09-13T10:34:33.546988+0000","last_became_peered":"2024-09-13T10:34:33.546988+0000","last_unstale":"2024-09-13T10:34:45.056923+0000","last_undegraded":"2024-09-13T10:34:45.056923+0000","last_fullsized":"2024-09-13T10:34:45.056923+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:53.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":26,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"99'4","reported_seq":31,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.266364+0000","last_change":"2024-09-13T10:34:33.543995+0000","last_active":"2024-09-13T10:34:45.266364+0000","last_peered":"2024-09-13T10:34:45.266364+0000","last_clean":"2024-09-13T10:34:45.266364+0000","last_became_active":"2024-09-13T10:34:33.543568+0000","last_became_peered":"2024-09-13T10:34:33.543568+0000","last_unstale":"2024-09-13T10:34:45.266364+0000","last_undegraded":"2024-09-13T10:34:45.266364+0000","last_fullsized":"2024-09-13T10:34:45.266364+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:37.712629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4],"acting":[9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"100'1","reported_seq":29,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.230596+0000","last_change":"2024-09-13T10:34:33.544186+0000","last_active":"2024-09-13T10:34:45.230596+0000","last_peered":"2024-09-13T10:34:45.230596+0000","last_clean":"2024-09-13T10:34:45.230596+0000","last_became_active":"2024-09-13T10:34:33.543949+0000","last_became_peered":"2024-09-13T10:34:33.543949+0000","last_unstale":"2024-09-13T10:34:45.230596+0000","last_undegraded":"2024-09-13T10:34:45.230596+0000","last_fullsized":"2024-09-13T10:34:45.230596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:33.283173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.056830+0000","last_change":"2024-09-13T10:34:33.546094+0000","last_active":"2024-09-13T10:34:45.056830+0000","last_peered":"2024-09-13T10:34:45.056830+0000","last_clean":"2024-09-13T10:34:45.056830+0000","last_became_active":"2024-09-13T10:34:33.545777+0000","last_became_peered":"2024-09-13T10:34:33.545777+0000","last_unstale":"2024-09-13T10:34:45.056830+0000","last_undegraded":"2024-09-13T10:34:45.056830+0000","last_fullsized":"2024-09-13T10:34:45.056830+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:55.844306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8],"acting":[6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":26,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-13T10:34:45.731325+0000","last_change":"2024-09-13T10:34:33.546634+0000","last_active":"2024-09-13T10:34:45.731325+0000","last_peered":"2024-09-13T10:34:45.731325+0000","last_clean":"2024-09-13T10:34:45.731325+0000","last_became_active":"2024-09-13T10:34:33.546326+0000","last_became_peered":"2024-09-13T10:34:33.546326+0000","last_unstale":"2024-09-13T10:34:45.731325+0000","last_undegraded":"2024-09-13T10:34:45.731325+0000","last_fullsized":"2024-09-13T10:34:45.731325+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:34:32.527188+0000","last_clean_scrub_stamp":"2024-09-13T10:34:32.527188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:57.664503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9],"acting":[11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":374,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3198,"num_read_kb":11287,"num_write":1940,"num_write_kb":6008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":38,"num_keys_recovered":26,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1403,"ondisk_log_size":1403,"up":64,"acting":64,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3727360,"data_stored":3702496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":7}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1404928,"data_stored":999707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481815,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":872448,"data_stored":470779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743454,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":888832,"data_stored":471073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005091,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":470799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299433,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1404928,"data_stored":999727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":876544,"data_stored":470841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":470799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":884736,"data_stored":471031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117053,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":470861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313281,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":470837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":888832,"data_stored":478581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":880640,"data_stored":470857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":58,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":42,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":62,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":42,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:34:52.551 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-13T10:34:52.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pg_num 11 2024-09-13T10:34:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:52.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:52.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.621 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:52.621 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:52.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:52.749 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:52.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:52.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:52.876 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:52.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:52.877 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:52.978 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:53.004 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:53.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:53.106 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:53.122 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:53 smithi060 ceph-mon[27375]: pgmap v328: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 198 KiB/s wr, 462 op/s; 5 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:53.129 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:53.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.130 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:53.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:53.257 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:53.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:53.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:53 smithi138 ceph-mon[31033]: pgmap v328: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 198 KiB/s wr, 462 op/s; 5 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:53.353 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T10:34:53.353 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:34:53.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:53.381 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:53.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.382 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:53.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:53.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:53.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:53.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:53 smithi173 ceph-mon[30864]: pgmap v328: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 198 KiB/s wr, 462 op/s; 5 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:53.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:53.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:53.681 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-13T10:34:53.681 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-13T10:34:53.681 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-13T10:34:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:34:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:34:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8: {} 2024-09-13T10:34:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9: {} 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10: {} 2024-09-13T10:34:53.683 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11: {} 2024-09-13T10:34:53.684 INFO:teuthology.orchestra.run.smithi060.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:34:53.684 INFO:teuthology.orchestra.run.smithi060.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:34:53.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:53.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.760 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:53.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:53.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:53.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-13T10:34:53.886 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:53.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:53.887 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:53.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:54.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.014 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:54.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:54.116 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:54.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.142 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:54.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:54.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:54.267 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:54.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:54.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:54 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3739320691' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:54.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:54 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3739320691' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:54.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-13T10:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:54.370 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:54.394 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:54.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.395 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:54.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:54.518 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:54.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.519 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:54.526 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:54 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3739320691' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:54.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:54.642 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:54.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.643 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:54.744 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:54.765 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:54.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.766 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:54.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:54.893 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:54.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:54.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:54.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:55.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:55.121 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:55.146 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:55.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:55.248 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:55.270 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:55.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.271 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:55.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:55 smithi138 ceph-mon[31033]: pgmap v329: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 85 KiB/s wr, 197 op/s; 4 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:55.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:55 smithi138 ceph-mon[31033]: osdmap e109: 12 total, 12 up, 12 in 2024-09-13T10:34:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:55.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:55 smithi060 ceph-mon[27375]: pgmap v329: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 85 KiB/s wr, 197 op/s; 4 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:55.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:55 smithi060 ceph-mon[27375]: osdmap e109: 12 total, 12 up, 12 in 2024-09-13T10:34:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:55.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:55.428 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:55 smithi173 ceph-mon[30864]: pgmap v329: 33 pgs: 33 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 85 KiB/s wr, 197 op/s; 4 B/s, 3 keys/s, 0 objects/s recovering 2024-09-13T10:34:55.428 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:55 smithi173 ceph-mon[30864]: osdmap e109: 12 total, 12 up, 12 in 2024-09-13T10:34:55.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:55.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.522 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:55.522 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:55.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:55.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:55.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:55.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:34:55.774 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:55.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:34:55.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:55.898 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:34:55.898 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T10:34:55.898 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T10:34:55.898 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T10:34:55.898 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:34:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:55.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:55.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:56.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:56.025 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:56.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.026 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pg_num to 11 2024-09-13T10:34:56.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:56.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:56.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:56.256 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:56.278 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:56.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.279 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:56.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:56 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:34:56.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:56 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:34:56.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:56 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2674903663' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-13T10:34:56.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:56 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:34:56.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:56 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:34:56.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:56 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2674903663' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-13T10:34:56.380 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:34:56.401 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:56.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.402 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:34:56.503 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:56.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.525 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:56.525 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:56.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:56 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:34:56.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:56 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:34:56.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:56 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2674903663' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-13T10:34:56.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:56.649 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:56.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:56.751 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:56.772 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:56.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.773 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:56.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:34:56.874 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:56.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:56.896 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:56.896 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:56.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:57.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:57.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:57.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:57.147 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:57.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:57.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:57.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:57.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:57.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:57 smithi138 ceph-mon[31033]: pgmap v331: 33 pgs: 33 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 248 KiB/s wr, 578 op/s 2024-09-13T10:34:57.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:57 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2674903663' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-13T10:34:57.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:57 smithi138 ceph-mon[31033]: osdmap e110: 12 total, 12 up, 12 in 2024-09-13T10:34:57.369 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:57.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:57 smithi060 ceph-mon[27375]: pgmap v331: 33 pgs: 33 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 248 KiB/s wr, 578 op/s 2024-09-13T10:34:57.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:57 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2674903663' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-13T10:34:57.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:57 smithi060 ceph-mon[27375]: osdmap e110: 12 total, 12 up, 12 in 2024-09-13T10:34:57.373 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:57.395 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:57.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.396 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:57.497 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:57.519 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:57.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.520 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:57.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:57 smithi173 ceph-mon[30864]: pgmap v331: 33 pgs: 33 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 248 KiB/s wr, 578 op/s 2024-09-13T10:34:57.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:57 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2674903663' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-13T10:34:57.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:57 smithi173 ceph-mon[30864]: osdmap e110: 12 total, 12 up, 12 in 2024-09-13T10:34:57.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:34:57.642 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:57.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.643 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:34:57.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:34:57.770 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:57.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:34:57.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:57.897 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:57.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:57.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:57.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-13T10:34:57.999 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:58.021 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:58.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.022 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:58.094 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-13T10:34:58.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:34:58.145 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:58.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.146 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:34:58.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:58.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.275 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:58.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:58.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2405161626' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:34:58.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:58 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-13T10:34:58.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:58 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2405161626' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:34:58.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:58 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-13T10:34:58.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:58.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:58.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:34:58.523 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:58.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:34:58.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:58 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2405161626' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:34:58.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:58 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-13T10:34:58.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:58.647 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:58.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:58.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:58.774 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:58.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:58.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:34:58.901 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:34:58.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:58.902 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:34:59.003 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:59.029 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:59.029 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.mirror_image_status (21367 ms) 2024-09-13T10:34:59.030 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.mirror_image_map 2024-09-13T10:34:59.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:59.044 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-13T10:34:59.044 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:34:59.066 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-13T10:34:59.067 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:34:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:34:59.088 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-13T10:34:59.089 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:34:59.111 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-13T10:34:59.111 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:34:59.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:59.132 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-13T10:34:59.133 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:34:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:59.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:59.155 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-13T10:34:59.155 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:34:59.176 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-13T10:34:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:34:59.197 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-13T10:34:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:34:59.219 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8: {} 2024-09-13T10:34:59.219 INFO:teuthology.orchestra.run.smithi060.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:34:59.243 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9: {} 2024-09-13T10:34:59.243 INFO:teuthology.orchestra.run.smithi060.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:34:59.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:34:59.265 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10: {} 2024-09-13T10:34:59.265 INFO:teuthology.orchestra.run.smithi060.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:34:59.279 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:59.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:34:59.287 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11: {} 2024-09-13T10:34:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:34:59.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: pgmap v333: 33 pgs: 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 117 KiB/s wr, 273 op/s 2024-09-13T10:34:59.289 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-13T10:34:59.289 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: Cluster is now healthy 2024-09-13T10:34:59.289 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2405161626' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-13T10:34:59.289 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-13T10:34:59.289 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: osdmap e111: 12 total, 12 up, 12 in 2024-09-13T10:34:59.289 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:34:59 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/695579579' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:59.338 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.mirror_image_map (309 ms) 2024-09-13T10:34:59.338 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.mirror_instances 2024-09-13T10:34:59.349 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.mirror_instances (11 ms) 2024-09-13T10:34:59.349 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.mirror_snapshot 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: pgmap v333: 33 pgs: 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 117 KiB/s wr, 273 op/s 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: Cluster is now healthy 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2405161626' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: osdmap e111: 12 total, 12 up, 12 in 2024-09-13T10:34:59.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:34:59 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/695579579' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:59.366 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.mirror_snapshot (15 ms) 2024-09-13T10:34:59.366 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_dir_list 2024-09-13T10:34:59.366 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_dir_list (3 ms) 2024-09-13T10:34:59.366 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_dir_add 2024-09-13T10:34:59.368 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_dir_add (2 ms) 2024-09-13T10:34:59.369 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.dir_add_already_existing 2024-09-13T10:34:59.371 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.dir_add_already_existing (3 ms) 2024-09-13T10:34:59.371 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_dir_rename 2024-09-13T10:34:59.377 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_dir_rename (6 ms) 2024-09-13T10:34:59.377 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_dir_remove 2024-09-13T10:34:59.380 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_dir_remove (3 ms) 2024-09-13T10:34:59.380 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_dir_remove_missing 2024-09-13T10:34:59.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:34:59.384 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_dir_remove_missing (4 ms) 2024-09-13T10:34:59.384 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_image_add 2024-09-13T10:34:59.387 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_image_add (2 ms) 2024-09-13T10:34:59.387 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_image_remove 2024-09-13T10:34:59.391 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_image_remove (4 ms) 2024-09-13T10:34:59.391 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_image_list 2024-09-13T10:34:59.393 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_image_list (3 ms) 2024-09-13T10:34:59.394 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_image_clean 2024-09-13T10:34:59.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:59.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:34:59.408 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_image_clean (11 ms) 2024-09-13T10:34:59.408 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.image_group_add 2024-09-13T10:34:59.408 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.image_group_add (2 ms) 2024-09-13T10:34:59.408 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.image_group_remove 2024-09-13T10:34:59.411 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.image_group_remove (4 ms) 2024-09-13T10:34:59.411 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.image_group_get 2024-09-13T10:34:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.image_group_get (3 ms) 2024-09-13T10:34:59.414 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_name 2024-09-13T10:34:59.416 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_set_empty_name (2 ms) 2024-09-13T10:34:59.416 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_id 2024-09-13T10:34:59.418 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_set_empty_id (2 ms) 2024-09-13T10:34:59.418 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_id 2024-09-13T10:34:59.420 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_id (3 ms) 2024-09-13T10:34:59.420 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_name 2024-09-13T10:34:59.424 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_name (3 ms) 2024-09-13T10:34:59.424 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_set 2024-09-13T10:34:59.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3 2024-09-13T10:34:59.508 INFO:teuthology.orchestra.run.smithi173.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.4-843-g2c01458a/rpm/el9/BUILD/ceph-18.2.4-843-g2c01458a/src/test/cls_rbd/test_cls_rbd.cc:2681: Failure 2024-09-13T10:34:59.508 INFO:teuthology.orchestra.run.smithi173.stdout:Expected equality of these values: 2024-09-13T10:34:59.508 INFO:teuthology.orchestra.run.smithi173.stdout: 1U 2024-09-13T10:34:59.508 INFO:teuthology.orchestra.run.smithi173.stdout: Which is: 1 2024-09-13T10:34:59.508 INFO:teuthology.orchestra.run.smithi173.stdout: keys.size() 2024-09-13T10:34:59.508 INFO:teuthology.orchestra.run.smithi173.stdout: Which is: 3 2024-09-13T10:34:59.509 INFO:teuthology.orchestra.run.smithi173.stdout:[ FAILED ] TestClsRbd.group_snap_set (84 ms) 2024-09-13T10:34:59.509 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_list 2024-09-13T10:34:59.511 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_list (3 ms) 2024-09-13T10:34:59.511 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_list_max_return 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: pgmap v333: 33 pgs: 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 117 KiB/s wr, 273 op/s 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: Cluster is now healthy 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2405161626' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: osdmap e111: 12 total, 12 up, 12 in 2024-09-13T10:34:59.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:34:59 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/695579579' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:34:59.528 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_list_max_return (17 ms) 2024-09-13T10:34:59.528 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_list_max_read 2024-09-13T10:34:59.530 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:34:59.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.3' 2024-09-13T10:34:59.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:34:59.656 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:59.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:34:59.758 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:59.780 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:59.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:34:59.881 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:34:59.904 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:34:59.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:34:59.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:35:00.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:35:00.029 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:00.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:35:00.131 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:35:00.158 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:00.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.159 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:35:00.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:35:00.282 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:00.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:35:00.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:00 smithi060 ceph-mon[27375]: osdmap e112: 12 total, 12 up, 12 in 2024-09-13T10:35:00.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:00 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:35:00.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:00 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:35:00.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-13T10:35:00.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:00 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-13T10:35:00.297 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:00 smithi060 ceph-mon[27375]: osdmap e113: 12 total, 12 up, 12 in 2024-09-13T10:35:00.316 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T10:35:00.316 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:35:00.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:00 smithi138 ceph-mon[31033]: osdmap e112: 12 total, 12 up, 12 in 2024-09-13T10:35:00.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:00 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:35:00.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:00 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:35:00.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-13T10:35:00.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:00 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-13T10:35:00.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:00 smithi138 ceph-mon[31033]: osdmap e113: 12 total, 12 up, 12 in 2024-09-13T10:35:00.385 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:35:00.411 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:00.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.412 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:35:00.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:35:00.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:00 smithi173 ceph-mon[30864]: osdmap e112: 12 total, 12 up, 12 in 2024-09-13T10:35:00.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:00 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:35:00.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:00 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:35:00.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-13T10:35:00.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:00 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-13T10:35:00.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:00 smithi173 ceph-mon[30864]: osdmap e113: 12 total, 12 up, 12 in 2024-09-13T10:35:00.537 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:00.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:35:00.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:35:00.666 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:00.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.667 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:35:00.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11 2024-09-13T10:35:00.793 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:00.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.794 INFO:tasks.cephadm.osd.11:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.11' 2024-09-13T10:35:00.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-13T10:35:00.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:35:00.920 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:00.921 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:35:00.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-13T10:35:01.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:35:01.048 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:01.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:35:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:01.150 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:35:01.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.172 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:01.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:35:01.215 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:01 smithi060 ceph-mon[27375]: pgmap v336: 43 pgs: 1 peering, 10 unknown, 32 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 165 KiB/s wr, 384 op/s 2024-09-13T10:35:01.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:35:01.299 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:01.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.300 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:35:01.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0 2024-09-13T10:35:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:01.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.0' 2024-09-13T10:35:01.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:35:01.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:01 smithi173 ceph-mon[30864]: pgmap v336: 43 pgs: 1 peering, 10 unknown, 32 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 165 KiB/s wr, 384 op/s 2024-09-13T10:35:01.548 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:01.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:35:01.565 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_list_max_read (2037 ms) 2024-09-13T10:35:01.565 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_remove 2024-09-13T10:35:01.567 INFO:teuthology.orchestra.run.smithi173.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.2.4-843-g2c01458a/rpm/el9/BUILD/ceph-18.2.4-843-g2c01458a/src/test/cls_rbd/test_cls_rbd.cc:2792: Failure 2024-09-13T10:35:01.567 INFO:teuthology.orchestra.run.smithi173.stdout:Expected equality of these values: 2024-09-13T10:35:01.567 INFO:teuthology.orchestra.run.smithi173.stdout: 1U 2024-09-13T10:35:01.568 INFO:teuthology.orchestra.run.smithi173.stdout: Which is: 1 2024-09-13T10:35:01.568 INFO:teuthology.orchestra.run.smithi173.stdout: keys.size() 2024-09-13T10:35:01.568 INFO:teuthology.orchestra.run.smithi173.stdout: Which is: 3 2024-09-13T10:35:01.568 INFO:teuthology.orchestra.run.smithi173.stdout:[ FAILED ] TestClsRbd.group_snap_remove (3 ms) 2024-09-13T10:35:01.568 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.group_snap_get_by_id 2024-09-13T10:35:01.569 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.group_snap_get_by_id (2 ms) 2024-09-13T10:35:01.569 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.trash_methods 2024-09-13T10:35:01.578 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.trash_methods (8 ms) 2024-09-13T10:35:01.578 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.op_features 2024-09-13T10:35:01.585 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.op_features (8 ms) 2024-09-13T10:35:01.585 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.clone_parent 2024-09-13T10:35:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:01 smithi138 ceph-mon[31033]: pgmap v336: 43 pgs: 1 peering, 10 unknown, 32 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 165 KiB/s wr, 384 op/s 2024-09-13T10:35:01.612 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.clone_parent (27 ms) 2024-09-13T10:35:01.612 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.clone_parent_ns 2024-09-13T10:35:01.622 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.clone_parent_ns (9 ms) 2024-09-13T10:35:01.622 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.clone_child 2024-09-13T10:35:01.635 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.clone_child (14 ms) 2024-09-13T10:35:01.635 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.namespace_methods 2024-09-13T10:35:01.643 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.namespace_methods (7 ms) 2024-09-13T10:35:01.643 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.migration 2024-09-13T10:35:01.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:35:01.655 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.migration (13 ms) 2024-09-13T10:35:01.655 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.migration_v1 2024-09-13T10:35:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:01.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:35:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:01.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:35:01.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.798 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:01.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:35:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.migration_v1 (144 ms) 2024-09-13T10:35:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.assert_snapc_seq 2024-09-13T10:35:01.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1 2024-09-13T10:35:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:01.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:01.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.1' 2024-09-13T10:35:01.977 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-13T10:35:01.978 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:35:01.978 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-13T10:35:01.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:35:02.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:35:02.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.046 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:02.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:35:02.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:35:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-13T10:35:02.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: osdmap e114: 12 total, 12 up, 12 in 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.4 scrub starts 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.4 scrub ok 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.9 scrub starts 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.9 scrub ok 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.3 scrub starts 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.3 scrub ok 2024-09-13T10:35:02.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.7 scrub starts 2024-09-13T10:35:02.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: 1.7 scrub ok 2024-09-13T10:35:02.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:02 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2385181659' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-13T10:35:02.147 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:35:02.172 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:02.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.173 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:35:02.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2 2024-09-13T10:35:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-13T10:35:02.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.2' 2024-09-13T10:35:02.307 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T10:35:02.308 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:35:02.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:35:02.424 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:02.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.425 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:35:02.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:35:02.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: osdmap e114: 12 total, 12 up, 12 in 2024-09-13T10:35:02.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.4 scrub starts 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.4 scrub ok 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.9 scrub starts 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.9 scrub ok 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.3 scrub starts 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.3 scrub ok 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.7 scrub starts 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: 1.7 scrub ok 2024-09-13T10:35:02.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:02 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2385181659' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-13T10:35:02.553 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:02.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.554 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:35:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: osdmap e114: 12 total, 12 up, 12 in 2024-09-13T10:35:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.4 scrub starts 2024-09-13T10:35:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.4 scrub ok 2024-09-13T10:35:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.9 scrub starts 2024-09-13T10:35:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.9 scrub ok 2024-09-13T10:35:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.3 scrub starts 2024-09-13T10:35:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.3 scrub ok 2024-09-13T10:35:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.7 scrub starts 2024-09-13T10:35:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: 1.7 scrub ok 2024-09-13T10:35:02.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:02 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2385181659' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-13T10:35:02.655 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5 2024-09-13T10:35:02.677 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:02.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.5' 2024-09-13T10:35:02.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:35:02.801 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:02.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:35:02.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:35:02.924 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:02.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:02.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:35:03.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:35:03.051 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:03.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.052 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:35:03.121 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.assert_snapc_seq (1321 ms) 2024-09-13T10:35:03.121 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN ] TestClsRbd.sparsify 2024-09-13T10:35:03.151 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:03.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:35:03.157 INFO:teuthology.orchestra.run.smithi173.stdout:[ OK ] TestClsRbd.sparsify (36 ms) 2024-09-13T10:35:03.178 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:03.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.179 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: 1.a scrub starts 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: 1.a scrub ok 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: 1.6 scrub starts 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: 1.6 scrub ok 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: pgmap v339: 43 pgs: 3 peering, 6 unknown, 34 active+clean; 844 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 421 KiB/s wr, 1.80k op/s 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2385181659' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: osdmap e115: 12 total, 12 up, 12 in 2024-09-13T10:35:03.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:03 smithi060 ceph-mon[27375]: 1.2 scrub starts 2024-09-13T10:35:03.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8 2024-09-13T10:35:03.307 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:03.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.307 INFO:tasks.cephadm.osd.8:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.8' 2024-09-13T10:35:03.408 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10 2024-09-13T10:35:03.433 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:03.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.434 INFO:tasks.cephadm.osd.10:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.10' 2024-09-13T10:35:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: 1.a scrub starts 2024-09-13T10:35:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: 1.a scrub ok 2024-09-13T10:35:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: 1.6 scrub starts 2024-09-13T10:35:03.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: 1.6 scrub ok 2024-09-13T10:35:03.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: pgmap v339: 43 pgs: 3 peering, 6 unknown, 34 active+clean; 844 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 421 KiB/s wr, 1.80k op/s 2024-09-13T10:35:03.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:35:03.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2385181659' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-13T10:35:03.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: osdmap e115: 12 total, 12 up, 12 in 2024-09-13T10:35:03.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:03 smithi173 ceph-mon[30864]: 1.2 scrub starts 2024-09-13T10:35:03.535 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:35:03.560 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:03.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.561 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: 1.a scrub starts 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: 1.a scrub ok 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: 1.6 scrub starts 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: 1.6 scrub ok 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: pgmap v339: 43 pgs: 3 peering, 6 unknown, 34 active+clean; 844 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 421 KiB/s wr, 1.80k op/s 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-13T10:35:03.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2385181659' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-13T10:35:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: osdmap e115: 12 total, 12 up, 12 in 2024-09-13T10:35:03.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:03 smithi138 ceph-mon[31033]: 1.2 scrub starts 2024-09-13T10:35:03.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-13T10:35:03.661 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6 2024-09-13T10:35:03.685 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:03.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.6' 2024-09-13T10:35:03.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9 2024-09-13T10:35:03.813 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: docker: command not found 2024-09-13T10:35:03.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.813 INFO:tasks.cephadm.osd.9:Ignoring exception while sending signal: Command failed on smithi173 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.9' 2024-09-13T10:35:03.859 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:03.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4 2024-09-13T10:35:03.940 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:03.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:03.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.4' 2024-09-13T10:35:04.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7 2024-09-13T10:35:04.067 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: docker: command not found 2024-09-13T10:35:04.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:04.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi138 with status 1: 'sudo docker kill -s 1 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd.7' 2024-09-13T10:35:04.133 INFO:teuthology.orchestra.run.smithi173.stdout:[----------] 68 tests from TestClsRbd (29616 ms total) 2024-09-13T10:35:04.133 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout:[----------] Global test environment tear-down 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout:[==========] 68 tests from 1 test suite ran. (31809 ms total) 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout:[ PASSED ] 66 tests. 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout:[ FAILED ] 2 tests, listed below: 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout:[ FAILED ] TestClsRbd.group_snap_set 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout:[ FAILED ] TestClsRbd.group_snap_remove 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-13T10:35:04.134 INFO:teuthology.orchestra.run.smithi173.stdout: 2 FAILED TESTS 2024-09-13T10:35:04.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:35:04.138 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi173 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" 2024-09-13T10:35:04.380 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5d8efaf651ad44dd8898cd78302af9d0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi173 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" 2024-09-13T10:35:04.382 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T10:35:04.390 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T10:35:04.394 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:35:04.396 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:35:04.396 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:04 smithi060 ceph-mon[27375]: 1.8 scrub starts 2024-09-13T10:35:04.397 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:04 smithi060 ceph-mon[27375]: 1.1 scrub starts 2024-09-13T10:35:04.397 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:04 smithi060 ceph-mon[27375]: osdmap e116: 12 total, 12 up, 12 in 2024-09-13T10:35:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:04 smithi173 ceph-mon[30864]: 1.8 scrub starts 2024-09-13T10:35:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:04 smithi173 ceph-mon[30864]: 1.1 scrub starts 2024-09-13T10:35:04.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:04 smithi173 ceph-mon[30864]: osdmap e116: 12 total, 12 up, 12 in 2024-09-13T10:35:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:04 smithi138 ceph-mon[31033]: 1.8 scrub starts 2024-09-13T10:35:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:04 smithi138 ceph-mon[31033]: 1.1 scrub starts 2024-09-13T10:35:04.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:04 smithi138 ceph-mon[31033]: osdmap e116: 12 total, 12 up, 12 in 2024-09-13T10:35:04.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-13T10:35:05.044 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-13T10:35:04.126600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649699,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":68,"num_write_kb":1174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":865519,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1012,"ondisk_log_size":1012,"up":22,"acting":22,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361580,"kb_used_data":12252,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580244,"statfs":{"total":1151940427776,"available":1151570169856,"internally_reserved":0,"allocated":12546048,"data_stored":7250917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":58957,"num_objects":-31,"num_object_clones":0,"num_object_copies":-62,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-31,"num_whiteouts":0,"num_read":-4314,"num_read_kb":-12398,"num_write":-2806,"num_write_kb":-7795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-7,"num_bytes_recovered":-721303,"num_keys_recovered":-26,"num_objects_omap":-24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.067921"},"pg_stats":[{"pgid":"1.8","version":"16'92","reported_seq":2,"reported_epoch":113,"state":"peering","last_fresh":"2024-09-13T10:34:59.105186+0000","last_change":"2024-09-13T10:35:00.657475+0000","last_active":"2024-09-13T10:35:00.657475+0000","last_peered":"2024-09-13T10:34:59.105186+0000","last_clean":"2024-09-13T10:34:59.105186+0000","last_became_active":"2024-09-13T10:34:59.103974+0000","last_became_peered":"2024-09-13T10:34:59.103974+0000","last_unstale":"2024-09-13T10:34:59.105186+0000","last_undegraded":"2024-09-13T10:34:59.105186+0000","last_fullsized":"2024-09-13T10:34:59.105186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":144253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'92","reported_seq":205,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T10:35:03.327140+0000","last_change":"2024-09-13T10:35:01.315897+0000","last_active":"2024-09-13T10:35:03.327140+0000","last_peered":"2024-09-13T10:35:03.327140+0000","last_clean":"2024-09-13T10:35:03.327140+0000","last_became_active":"2024-09-13T10:35:01.108329+0000","last_became_peered":"2024-09-13T10:35:01.108329+0000","last_unstale":"2024-09-13T10:35:03.327140+0000","last_undegraded":"2024-09-13T10:35:03.327140+0000","last_fullsized":"2024-09-13T10:35:03.327140+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:01.315852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:01.315852+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:11.094355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"16'92","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:35:01.108790+0000","last_change":"2024-09-13T10:35:01.107827+0000","last_active":"2024-09-13T10:35:01.108790+0000","last_peered":"2024-09-13T10:35:01.108790+0000","last_clean":"2024-09-13T10:35:01.108790+0000","last_became_active":"2024-09-13T10:35:01.107379+0000","last_became_peered":"2024-09-13T10:35:01.107379+0000","last_unstale":"2024-09-13T10:35:01.108790+0000","last_undegraded":"2024-09-13T10:35:01.108790+0000","last_fullsized":"2024-09-13T10:35:01.108790+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":11,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":144253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6],"acting":[9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"16'92","reported_seq":123,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:35:01.305658+0000","last_change":"2024-09-13T10:35:01.305592+0000","last_active":"2024-09-13T10:35:01.305658+0000","last_peered":"2024-09-13T10:35:01.305658+0000","last_clean":"2024-09-13T10:35:01.305658+0000","last_became_active":"2024-09-13T10:35:01.110688+0000","last_became_peered":"2024-09-13T10:35:01.110688+0000","last_unstale":"2024-09-13T10:35:01.305658+0000","last_undegraded":"2024-09-13T10:35:01.305658+0000","last_fullsized":"2024-09-13T10:35:01.305658+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:01.305517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:01.305517+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:07.170772+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8],"acting":[11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.5","version":"16'92","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:35:02.372558+0000","last_change":"2024-09-13T10:35:01.488141+0000","last_active":"2024-09-13T10:35:02.372558+0000","last_peered":"2024-09-13T10:35:02.372558+0000","last_clean":"2024-09-13T10:35:02.372558+0000","last_became_active":"2024-09-13T10:35:01.487873+0000","last_became_peered":"2024-09-13T10:35:01.487873+0000","last_unstale":"2024-09-13T10:35:02.372558+0000","last_undegraded":"2024-09-13T10:35:02.372558+0000","last_fullsized":"2024-09-13T10:35:02.372558+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":144253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2],"acting":[10,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.6","version":"16'92","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:35:02.372714+0000","last_change":"2024-09-13T10:35:01.504057+0000","last_active":"2024-09-13T10:35:02.372714+0000","last_peered":"2024-09-13T10:35:02.372714+0000","last_clean":"2024-09-13T10:35:02.372714+0000","last_became_active":"2024-09-13T10:35:01.108810+0000","last_became_peered":"2024-09-13T10:35:01.108810+0000","last_unstale":"2024-09-13T10:35:02.372714+0000","last_undegraded":"2024-09-13T10:35:02.372714+0000","last_fullsized":"2024-09-13T10:35:02.372714+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:01.503992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:01.503992+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:16.033546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9],"acting":[10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.7","version":"16'92","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:35:01.718945+0000","last_change":"2024-09-13T10:35:01.718889+0000","last_active":"2024-09-13T10:35:01.718945+0000","last_peered":"2024-09-13T10:35:01.718945+0000","last_clean":"2024-09-13T10:35:01.718945+0000","last_became_active":"2024-09-13T10:35:01.107260+0000","last_became_peered":"2024-09-13T10:35:01.107260+0000","last_unstale":"2024-09-13T10:35:01.718945+0000","last_undegraded":"2024-09-13T10:35:01.718945+0000","last_fullsized":"2024-09-13T10:35:01.718945+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:01.718850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:01.718850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.239350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"16'92","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:35:02.507935+0000","last_change":"2024-09-13T10:35:03.267254+0000","last_active":"2024-09-13T10:35:02.507935+0000","last_peered":"2024-09-13T10:35:02.507935+0000","last_clean":"2024-09-13T10:35:02.507935+0000","last_became_active":"2024-09-13T10:35:01.109898+0000","last_became_peered":"2024-09-13T10:35:01.109898+0000","last_unstale":"2024-09-13T10:35:02.507935+0000","last_undegraded":"2024-09-13T10:35:02.507935+0000","last_fullsized":"2024-09-13T10:35:02.507935+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-13T10:35:07.508006+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":12,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":144254,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9],"acting":[10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.2","version":"16'92","reported_seq":200,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T10:35:03.327037+0000","last_change":"2024-09-13T10:35:03.327037+0000","last_active":"2024-09-13T10:35:03.327037+0000","last_peered":"2024-09-13T10:35:03.327037+0000","last_clean":"2024-09-13T10:35:03.327037+0000","last_became_active":"2024-09-13T10:35:01.143556+0000","last_became_peered":"2024-09-13T10:35:01.143556+0000","last_unstale":"2024-09-13T10:35:03.327037+0000","last_undegraded":"2024-09-13T10:35:03.327037+0000","last_fullsized":"2024-09-13T10:35:03.327037+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-13T10:35:07.286030+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":12,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":144253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'92","reported_seq":2,"reported_epoch":113,"state":"peering","last_fresh":"2024-09-13T10:34:59.104391+0000","last_change":"2024-09-13T10:35:00.422533+0000","last_active":"2024-09-13T10:35:00.422533+0000","last_peered":"2024-09-13T10:34:59.104391+0000","last_clean":"2024-09-13T10:34:59.104391+0000","last_became_active":"2024-09-13T10:34:59.103396+0000","last_became_peered":"2024-09-13T10:34:59.103396+0000","last_unstale":"2024-09-13T10:34:59.104391+0000","last_undegraded":"2024-09-13T10:34:59.104391+0000","last_fullsized":"2024-09-13T10:34:59.104391+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:13.003638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:13.003638+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":144253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'92","reported_seq":200,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T10:35:03.327056+0000","last_change":"2024-09-13T10:34:59.376182+0000","last_active":"2024-09-13T10:35:03.327056+0000","last_peered":"2024-09-13T10:35:03.327056+0000","last_clean":"2024-09-13T10:35:03.327056+0000","last_became_active":"2024-09-13T10:34:59.101559+0000","last_became_peered":"2024-09-13T10:34:59.101559+0000","last_unstale":"2024-09-13T10:35:03.327056+0000","last_undegraded":"2024-09-13T10:35:03.327056+0000","last_fullsized":"2024-09-13T10:35:03.327056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:34:59.376132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:34:59.376132+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:28.684112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":649699,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":68,"num_write_kb":1174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":865519,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1012,"ondisk_log_size":1012,"up":22,"acting":22,"num_store_stats":10}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1503232,"data_stored":1062921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481818,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":983040,"data_stored":542073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":974848,"data_stored":526479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005094,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36632,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708520,"statfs":{"total":95995035648,"available":95957524480,"internally_reserved":0,"allocated":1314816,"data_stored":869764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":542081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":534043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":526205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":542313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117055,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":510409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313284,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":526243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40456,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704696,"statfs":{"total":95995035648,"available":95953608704,"internally_reserved":0,"allocated":962560,"data_stored":526247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":542139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":201248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:35:05.045 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-13T10:35:05.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pg_num 21 2024-09-13T10:35:05.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:05 smithi060 ceph-mon[27375]: 1.5 scrub starts 2024-09-13T10:35:05.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:05 smithi060 ceph-mon[27375]: pgmap v342: 43 pgs: 2 peering, 41 active+clean; 635 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 251 KiB/s wr, 1.86k op/s 2024-09-13T10:35:05.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:05 smithi060 ceph-mon[27375]: osdmap e117: 12 total, 12 up, 12 in 2024-09-13T10:35:05.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:05 smithi060 ceph-mon[27375]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:35:05.262 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-13T10:35:05.262 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:35:05.288 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-13T10:35:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:35:05.317 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-13T10:35:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:35:05.319 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:05.348 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-13T10:35:05.349 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:35:05.370 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-13T10:35:05.370 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:35:05.392 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-13T10:35:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:35:05.414 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-13T10:35:05.414 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:35:05.436 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-13T10:35:05.437 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:35:05.459 INFO:teuthology.orchestra.run.smithi060.stdout:osd.8: {} 2024-09-13T10:35:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:35:05.481 INFO:teuthology.orchestra.run.smithi060.stdout:osd.9: {} 2024-09-13T10:35:05.482 INFO:teuthology.orchestra.run.smithi060.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:35:05.503 INFO:teuthology.orchestra.run.smithi060.stdout:osd.10: {} 2024-09-13T10:35:05.503 INFO:teuthology.orchestra.run.smithi060.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:35:05.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.11: {} 2024-09-13T10:35:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:35:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:05 smithi173 ceph-mon[30864]: 1.5 scrub starts 2024-09-13T10:35:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:05 smithi173 ceph-mon[30864]: pgmap v342: 43 pgs: 2 peering, 41 active+clean; 635 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 251 KiB/s wr, 1.86k op/s 2024-09-13T10:35:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:05 smithi173 ceph-mon[30864]: osdmap e117: 12 total, 12 up, 12 in 2024-09-13T10:35:05.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:05 smithi173 ceph-mon[30864]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:35:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:05 smithi138 ceph-mon[31033]: 1.5 scrub starts 2024-09-13T10:35:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:05 smithi138 ceph-mon[31033]: pgmap v342: 43 pgs: 2 peering, 41 active+clean; 635 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 251 KiB/s wr, 1.86k op/s 2024-09-13T10:35:05.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:05 smithi138 ceph-mon[31033]: osdmap e117: 12 total, 12 up, 12 in 2024-09-13T10:35:05.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:05 smithi138 ceph-mon[31033]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:35:05.759 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:05.958 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:35:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T10:35:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T10:35:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T10:35:05.958 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:35:06.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:06 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2707103331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:35:06.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:06 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2707103331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:35:06.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:06 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2707103331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-13T10:35:06.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-13T10:35:07.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:07 smithi060 ceph-mon[27375]: pgmap v344: 11 pgs: 11 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:07.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:07 smithi173 ceph-mon[30864]: pgmap v344: 11 pgs: 11 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:07.576 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:07.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:07 smithi138 ceph-mon[31033]: pgmap v344: 11 pgs: 11 active+clean; 582 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pg_num to 21 2024-09-13T10:35:08.268 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T10:35:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T10:35:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T10:35:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T10:35:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:32 2.1e 2:7a018b73:::image4:head [call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.136230+0000", 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.130173435000003, 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00094941000000000003, 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 32 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136230+0000", 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136230+0000", 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136231+0000", 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8699999999999995e-07 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136234+0000", 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.264e-06 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136241+0000", 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2300000000000002e-06 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136246+0000", 2024-09-13T10:35:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.284e-06 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136286+0000", 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0803999999999998e-05 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.136382+0000", 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5674999999999995e-05 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137180+0000", 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00079766600000000004 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:33 2.1e 2:7a018b73:::image4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.137389+0000", 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.129015152999997, 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00088281899999999996, 2024-09-13T10:35:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 33 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137389+0000", 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137389+0000", 2024-09-13T10:35:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137389+0000", 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9699999999999995e-07 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137393+0000", 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4489999999999999e-06 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137403+0000", 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0489000000000001e-05 2024-09-13T10:35:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137408+0000", 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9910000000000002e-06 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137459+0000", 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1153000000000003e-05 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137529+0000", 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9618999999999999e-05 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0", 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137663+0000", 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013451700000000001 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.137932+0000", 2024-09-13T10:35:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026819700000000002 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138219+0000", 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028717100000000002 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138246+0000", 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7336999999999999e-05 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138271+0000", 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5199e-05 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:34 2.1e 2:7a018b73:::image4:head [call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.138563+0000", 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.127840212999999, 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067027899999999997, 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 34 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138563+0000", 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138563+0000", 2024-09-13T10:35:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138564+0000", 2024-09-13T10:35:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-07 2024-09-13T10:35:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138568+0000", 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2089999999999999e-06 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138579+0000", 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0190000000000001e-05 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138584+0000", 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0980000000000001e-06 2024-09-13T10:35:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138613+0000", 2024-09-13T10:35:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8994999999999999e-05 2024-09-13T10:35:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.138660+0000", 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6953999999999998e-05 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139234+0000", 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00057415700000000005 2024-09-13T10:35:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:35 2.1e 2:7a018b73:::image4:head [call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.139479+0000", 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.126924744999997, 2024-09-13T10:35:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072143299999999995, 2024-09-13T10:35:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T10:35:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 35 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139479+0000", 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139479+0000", 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139480+0000", 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3e-07 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139483+0000", 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6440000000000002e-06 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139493+0000", 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7189999999999996e-06 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139504+0000", 2024-09-13T10:35:08.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0805000000000001e-05 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139537+0000", 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3081999999999999e-05 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.139580+0000", 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2663000000000001e-05 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140200+0000", 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00062089000000000003 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:37 2.1e 2:7a018b73:::image4:head [call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.140658+0000", 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.125745414999997, 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000808894, 2024-09-13T10:35:08.304 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 37 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.305 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140658+0000", 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140658+0000", 2024-09-13T10:35:08.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140659+0000", 2024-09-13T10:35:08.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2200000000000004e-07 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140663+0000", 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0969999999999999e-06 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140671+0000", 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6190000000000002e-06 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140675+0000", 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0119999999999999e-06 2024-09-13T10:35:08.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140701+0000", 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6094000000000001e-05 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140738+0000", 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7165000000000001e-05 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0", 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.140828+0000", 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0197000000000004e-05 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141090+0000", 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026207999999999998 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141413+0000", 2024-09-13T10:35:08.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032299899999999997 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141441+0000", 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8062e-05 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141467+0000", 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5947000000000001e-05 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.314 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:38 2.1e 2:7a018b73:::image4:head [call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.141708+0000", 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.124695441999997, 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00070181300000000002, 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.315 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 38 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141708+0000", 2024-09-13T10:35:08.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141708+0000", 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141709+0000", 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2300000000000001e-07 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141712+0000", 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5360000000000001e-06 2024-09-13T10:35:08.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141721+0000", 2024-09-13T10:35:08.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5930000000000006e-06 2024-09-13T10:35:08.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141725+0000", 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4039999999999997e-06 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141753+0000", 2024-09-13T10:35:08.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7773e-05 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.141799+0000", 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6041999999999997e-05 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142410+0000", 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000610842 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:39 2.1e 2:7a018b73:::image4:head [call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.142599+0000", 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.123805062000002, 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00070943600000000003, 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142599+0000", 2024-09-13T10:35:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142599+0000", 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142599+0000", 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1099999999999995e-07 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142603+0000", 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.371e-06 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142611+0000", 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3659999999999992e-06 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142622+0000", 2024-09-13T10:35:08.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1413000000000001e-05 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142656+0000", 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3353999999999999e-05 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.142702+0000", 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6684000000000002e-05 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143308+0000", 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00060563700000000002 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.329 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24508.0:40 2.1e 2:7a018b73:::image4:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.143592+0000", 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.122811339999998, 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034588899999999999, 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24508", 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 40 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143592+0000", 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143592+0000", 2024-09-13T10:35:08.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143593+0000", 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6000000000000004e-07 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143602+0000", 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4520000000000003e-06 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143611+0000", 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7360000000000007e-06 2024-09-13T10:35:08.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143615+0000", 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2420000000000002e-06 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143667+0000", 2024-09-13T10:35:08.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1764000000000001e-05 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143754+0000", 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6582999999999994e-05 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.143938+0000", 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018455199999999999 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.337 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:106 2.1 e98/97 2:803aa790:::image10:head v 98'1, mlcod=0'0)", 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.205267+0000", 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.061137021999997, 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00038550999999999999, 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205267+0000", 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205267+0000", 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205267+0000", 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0900000000000001e-07 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205271+0000", 2024-09-13T10:35:08.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7749999999999999e-06 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205319+0000", 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7874999999999998e-05 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205326+0000", 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5909999999999998e-06 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205382+0000", 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6143000000000001e-05 2024-09-13T10:35:08.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205397+0000", 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4429e-05 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205622+0000", 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022570899999999999 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.205652+0000", 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9779000000000001e-05 2024-09-13T10:35:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:108 2.1 e98/97 2:803aa790:::image10:head v 98'3, mlcod=98'1)", 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.207115+0000", 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.059288956000003, 2024-09-13T10:35:08.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036599400000000002, 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207115+0000", 2024-09-13T10:35:08.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207115+0000", 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207115+0000", 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0100000000000005e-07 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207119+0000", 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0260000000000001e-06 2024-09-13T10:35:08.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207143+0000", 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4206000000000002e-05 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207148+0000", 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.245e-06 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207190+0000", 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2543999999999998e-05 2024-09-13T10:35:08.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207202+0000", 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1459e-05 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207454+0000", 2024-09-13T10:35:08.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025221400000000001 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.207481+0000", 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6798999999999999e-05 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:111 2.1 e98/97 2:803aa790:::image10:head v 98'6, mlcod=98'4)", 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.209892+0000", 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.056511899, 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042323399999999998, 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209892+0000", 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209892+0000", 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209892+0000", 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1999999999999999e-07 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209898+0000", 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4260000000000002e-06 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209931+0000", 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2972999999999998e-05 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209938+0000", 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4900000000000003e-06 2024-09-13T10:35:08.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.209995+0000", 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7148000000000002e-05 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210015+0000", 2024-09-13T10:35:08.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9854999999999999e-05 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210276+0000", 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026066700000000002 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210315+0000", 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9054999999999998e-05 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:112 2.1 e98/97 2:803aa790:::image10:head v 98'7, mlcod=98'5)", 2024-09-13T10:35:08.360 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.210876+0000", 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.055528127000002, 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000347463, 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210876+0000", 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210876+0000", 2024-09-13T10:35:08.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210876+0000", 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4199999999999995e-07 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210881+0000", 2024-09-13T10:35:08.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2379999999999997e-06 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210914+0000", 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2191999999999999e-05 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210921+0000", 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4440000000000002e-06 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210963+0000", 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2333999999999998e-05 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.210976+0000", 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2995e-05 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.211192+0000", 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215532 2024-09-13T10:35:08.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.211223+0000", 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1086e-05 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:115 2.1 e98/97 2:803aa790:::image10:head v 98'10, mlcod=98'8)", 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.213753+0000", 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.052650219, 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032424199999999998, 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213753+0000", 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213753+0000", 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213754+0000", 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5000000000000002e-07 2024-09-13T10:35:08.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213761+0000", 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5799999999999997e-06 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213792+0000", 2024-09-13T10:35:08.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1726000000000001e-05 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213800+0000", 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2640000000000002e-06 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213836+0000", 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6668999999999997e-05 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.213848+0000", 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1990999999999999e-05 2024-09-13T10:35:08.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.214054+0000", 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020605499999999999 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.214078+0000", 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3306999999999999e-05 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:124 2.e e98/97 2:70c7e61f:::image12:head v 98'1, mlcod=0'0)", 2024-09-13T10:35:08.375 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.222042+0000", 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.044361572, 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034357499999999999, 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222042+0000", 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222042+0000", 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222043+0000", 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3300000000000002e-07 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222048+0000", 2024-09-13T10:35:08.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.079e-06 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222075+0000", 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7058000000000001e-05 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222081+0000", 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4910000000000002e-06 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222134+0000", 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2506999999999997e-05 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222151+0000", 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7212999999999999e-05 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222362+0000", 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021097499999999999 2024-09-13T10:35:08.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.222386+0000", 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3618999999999999e-05 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:126 2.12 e98/97 2:48e4d0dc:::image13:head v 98'1, mlcod=0'0)", 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:34.228570+0000", 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.037834124, 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035097600000000002, 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228570+0000", 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228570+0000", 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228570+0000", 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.34e-07 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228588+0000", 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8182e-05 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228615+0000", 2024-09-13T10:35:08.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7149e-05 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228620+0000", 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7600000000000002e-06 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228670+0000", 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0133999999999998e-05 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228689+0000", 2024-09-13T10:35:08.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8567e-05 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228888+0000", 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019912099999999999 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:34.228921+0000", 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2829e-05 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:152 2.a e99/97 2:57642022:::image19:head v 99'1, mlcod=0'0)", 2024-09-13T10:35:08.390 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:35.240917+0000", 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 33.025486424, 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00065726599999999999, 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.240917+0000", 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.240917+0000", 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.240917+0000", 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.79e-07 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.240922+0000", 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9019999999999998e-06 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.240970+0000", 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7704999999999998e-05 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.240979+0000", 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8249999999999994e-06 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.241044+0000", 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4635999999999996e-05 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.241061+0000", 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7691000000000001e-05 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.241365+0000", 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030398699999999997 2024-09-13T10:35:08.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.241575+0000", 2024-09-13T10:35:08.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020934099999999999 2024-09-13T10:35:08.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.399 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:190 2.e e99/97 2:77547369:::image22:head v 99'2, mlcod=0'0)", 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:35.281399+0000", 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.985004207999999, 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00044206300000000003, 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281399+0000", 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281399+0000", 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281401+0000", 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.026e-06 2024-09-13T10:35:08.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281410+0000", 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5610000000000004e-06 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281455+0000", 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4527999999999998e-05 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281463+0000", 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4530000000000002e-06 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281527+0000", 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3757000000000006e-05 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281550+0000", 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3181e-05 2024-09-13T10:35:08.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281790+0000", 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023986799999999999 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.281842+0000", 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1688999999999999e-05 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24508.0:197 2.e e99/97 2:77547369:::image22:head v 99'7, mlcod=99'5)", 2024-09-13T10:35:08.407 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:35.292609+0000", 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.973794382000001, 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035636400000000002, 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292609+0000", 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292609+0000", 2024-09-13T10:35:08.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292610+0000", 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8100000000000003e-07 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292616+0000", 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3840000000000002e-06 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292647+0000", 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0244999999999999e-05 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292653+0000", 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2090000000000002e-06 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292698+0000", 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5448999999999998e-05 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292713+0000", 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4748e-05 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292917+0000", 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000203852 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.292966+0000", 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8896000000000001e-05 2024-09-13T10:35:08.414 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_update_log_missing(2.e epoch 99/97 rep_tid 22 entries 99'8 (0'0) error 2:77547369:::image22:head by client.24508.0:198 0.000000 -17 [r=-17+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 99'6)", 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:35.293585+0000", 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.972818893000003, 2024-09-13T10:35:08.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037050700000000001, 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293585+0000", 2024-09-13T10:35:08.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293585+0000", 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293585+0000", 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9400000000000005e-07 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293592+0000", 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0249999999999997e-06 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293633+0000", 2024-09-13T10:35:08.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0123999999999997e-05 2024-09-13T10:35:08.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293645+0000", 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2917e-05 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293729+0000", 2024-09-13T10:35:08.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3207999999999993e-05 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.293955+0000", 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000226639 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_update_log_missing(2.e epoch 99/97 rep_tid 25 entries 99'11 (0'0) error 2:77547369:::image22:head by client.24508.0:202 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 99'9)", 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T10:34:35.297245+0000", 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.969158450000002, 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032498599999999998, 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-13T10:35:08.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297245+0000", 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297245+0000", 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T10:35:08.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297246+0000", 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297253+0000", 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9600000000000003e-06 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297300+0000", 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6709000000000003e-05 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297314+0000", 2024-09-13T10:35:08.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4273e-05 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297397+0000", 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2794999999999996e-05 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T10:35:08.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T10:34:35.297570+0000", 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017360300000000001 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T10:35:08.428 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T10:35:08.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:08 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1643942427' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2024-09-13T10:35:08.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:08 smithi060 ceph-mon[27375]: 1.2 scrub starts 2024-09-13T10:35:08.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:08 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1643942427' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2024-09-13T10:35:08.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:08 smithi173 ceph-mon[30864]: 1.2 scrub starts 2024-09-13T10:35:08.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:08 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1643942427' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2024-09-13T10:35:08.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:08 smithi138 ceph-mon[31033]: 1.2 scrub starts 2024-09-13T10:35:09.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:09 smithi060 ceph-mon[27375]: 1.1 scrub starts 2024-09-13T10:35:09.447 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:09 smithi060 ceph-mon[27375]: pgmap v345: 11 pgs: 11 active+clean; 530 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:09.448 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:09 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1643942427' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-13T10:35:09.448 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:09 smithi060 ceph-mon[27375]: osdmap e118: 12 total, 12 up, 12 in 2024-09-13T10:35:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:09 smithi173 ceph-mon[30864]: 1.1 scrub starts 2024-09-13T10:35:09.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:09 smithi173 ceph-mon[30864]: pgmap v345: 11 pgs: 11 active+clean; 530 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:09.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:09 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1643942427' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-13T10:35:09.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:09 smithi173 ceph-mon[30864]: osdmap e118: 12 total, 12 up, 12 in 2024-09-13T10:35:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:09 smithi138 ceph-mon[31033]: 1.1 scrub starts 2024-09-13T10:35:09.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:09 smithi138 ceph-mon[31033]: pgmap v345: 11 pgs: 11 active+clean; 530 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:09.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:09 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1643942427' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-13T10:35:09.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:09 smithi138 ceph-mon[31033]: osdmap e118: 12 total, 12 up, 12 in 2024-09-13T10:35:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-13T10:35:10.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: 1.8 scrub starts 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1804955223' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: Cluster is now healthy 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1804955223' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-13T10:35:10.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:10 smithi060 ceph-mon[27375]: osdmap e119: 12 total, 12 up, 12 in 2024-09-13T10:35:10.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-13T10:35:10.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: 1.8 scrub starts 2024-09-13T10:35:10.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-13T10:35:10.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1804955223' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-13T10:35:10.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-13T10:35:10.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: Cluster is now healthy 2024-09-13T10:35:10.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-13T10:35:10.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1804955223' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-13T10:35:10.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:10 smithi173 ceph-mon[30864]: osdmap e119: 12 total, 12 up, 12 in 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: 1.8 scrub starts 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1804955223' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: Cluster is now healthy 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-13T10:35:10.532 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1804955223' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-13T10:35:10.533 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:10 smithi138 ceph-mon[31033]: osdmap e119: 12 total, 12 up, 12 in 2024-09-13T10:35:10.761 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:11.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-13T10:35:11.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:11 smithi138 ceph-mon[31033]: 1.5 scrub starts 2024-09-13T10:35:11.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:11 smithi138 ceph-mon[31033]: pgmap v347: 11 pgs: 2 active+clean+scrubbing, 9 active+clean; 530 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:11.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:11 smithi138 ceph-mon[31033]: osdmap e120: 12 total, 12 up, 12 in 2024-09-13T10:35:11.390 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:11.390 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:11.390 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-13T10:35:11.390 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:11 smithi173 ceph-mon[30864]: 1.5 scrub starts 2024-09-13T10:35:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:11 smithi173 ceph-mon[30864]: pgmap v347: 11 pgs: 2 active+clean+scrubbing, 9 active+clean; 530 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:11.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:11 smithi173 ceph-mon[30864]: osdmap e120: 12 total, 12 up, 12 in 2024-09-13T10:35:11.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:11 smithi060 ceph-mon[27375]: 1.5 scrub starts 2024-09-13T10:35:11.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:11 smithi060 ceph-mon[27375]: pgmap v347: 11 pgs: 2 active+clean+scrubbing, 9 active+clean; 530 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:11.573 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:11 smithi060 ceph-mon[27375]: osdmap e120: 12 total, 12 up, 12 in 2024-09-13T10:35:12.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-13T10:35:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.8 scrub starts 2024-09-13T10:35:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.8 scrub ok 2024-09-13T10:35:12.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.3 scrub starts 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.3 scrub ok 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.4 scrub starts 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.4 scrub ok 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:12.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.7 scrub starts 2024-09-13T10:35:12.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: 1.7 scrub ok 2024-09-13T10:35:12.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:12 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-13T10:35:12.474 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:12.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.8 scrub starts 2024-09-13T10:35:12.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.8 scrub ok 2024-09-13T10:35:12.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.3 scrub starts 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.3 scrub ok 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.4 scrub starts 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.4 scrub ok 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:12.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.7 scrub starts 2024-09-13T10:35:12.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: 1.7 scrub ok 2024-09-13T10:35:12.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:12 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-13T10:35:12.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.8 scrub starts 2024-09-13T10:35:12.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.8 scrub ok 2024-09-13T10:35:12.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.3 scrub starts 2024-09-13T10:35:12.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.3 scrub ok 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.4 scrub starts 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.4 scrub ok 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.7 scrub starts 2024-09-13T10:35:12.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: 1.7 scrub ok 2024-09-13T10:35:12.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:12 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-13T10:35:12.946 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-13T10:35:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-13T10:35:12.947 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-13T10:35:13.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: 1.5 scrub starts 2024-09-13T10:35:13.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: 1.5 scrub ok 2024-09-13T10:35:13.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: pgmap v350: 21 pgs: 1 peering, 8 unknown, 2 active+clean+scrubbing, 10 active+clean; 734 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:13.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-13T10:35:13.515 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: osdmap e121: 12 total, 12 up, 12 in 2024-09-13T10:35:13.516 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1498255745' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:35:13.516 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: 1.2 scrub starts 2024-09-13T10:35:13.516 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:13 smithi060 ceph-mon[27375]: 1.2 scrub ok 2024-09-13T10:35:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: 1.5 scrub starts 2024-09-13T10:35:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: 1.5 scrub ok 2024-09-13T10:35:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: pgmap v350: 21 pgs: 1 peering, 8 unknown, 2 active+clean+scrubbing, 10 active+clean; 734 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:13.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-13T10:35:13.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: osdmap e121: 12 total, 12 up, 12 in 2024-09-13T10:35:13.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1498255745' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:35:13.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: 1.2 scrub starts 2024-09-13T10:35:13.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:13 smithi173 ceph-mon[30864]: 1.2 scrub ok 2024-09-13T10:35:13.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: 1.5 scrub starts 2024-09-13T10:35:13.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: 1.5 scrub ok 2024-09-13T10:35:13.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: pgmap v350: 21 pgs: 1 peering, 8 unknown, 2 active+clean+scrubbing, 10 active+clean; 734 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:13.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-13T10:35:13.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: osdmap e121: 12 total, 12 up, 12 in 2024-09-13T10:35:13.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1498255745' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-13T10:35:13.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: 1.2 scrub starts 2024-09-13T10:35:13.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:13 smithi138 ceph-mon[31033]: 1.2 scrub ok 2024-09-13T10:35:13.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-13T10:35:13.958 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:14.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.1 scrub starts 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.1 scrub ok 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1498255745' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: osdmap e122: 12 total, 12 up, 12 in 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.13 scrub starts 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.13 scrub ok 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.d scrub starts 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.d scrub ok 2024-09-13T10:35:14.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.b scrub starts 2024-09-13T10:35:14.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.b scrub ok 2024-09-13T10:35:14.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.f scrub starts 2024-09-13T10:35:14.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.f scrub ok 2024-09-13T10:35:14.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.e scrub starts 2024-09-13T10:35:14.352 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:14 smithi138 ceph-mon[31033]: 1.e scrub ok 2024-09-13T10:35:14.380 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:136 2.11 e98/97 2:8dfbcd02:::image16:head v 98'1, mlcod=0'0)", 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.240190+0000", 2024-09-13T10:35:14.381 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.138258542999999, 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00047208999999999999, 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240190+0000", 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240190+0000", 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240190+0000", 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5599999999999999e-07 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240195+0000", 2024-09-13T10:35:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5960000000000003e-06 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240242+0000", 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6233000000000003e-05 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240262+0000", 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0373e-05 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240327+0000", 2024-09-13T10:35:14.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5325000000000002e-05 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240351+0000", 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3370000000000002e-05 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240625+0000", 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027391999999999998 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240662+0000", 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:35:14.388 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:138 2.11 e98/97 2:8dfbcd02:::image16:head v 98'2, mlcod=0'0)", 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.246814+0000", 2024-09-13T10:35:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.131634138999999, 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00044086800000000002, 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246814+0000", 2024-09-13T10:35:14.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246814+0000", 2024-09-13T10:35:14.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246815+0000", 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.37e-07 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246820+0000", 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4020000000000001e-06 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246851+0000", 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0939000000000003e-05 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246857+0000", 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4219999999999999e-06 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246911+0000", 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4450000000000002e-05 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246942+0000", 2024-09-13T10:35:14.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1324e-05 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247221+0000", 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027902999999999999 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247255+0000", 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3563999999999999e-05 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.397 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:139 2.11 e98/97 2:8dfbcd02:::image16:head v 98'3, mlcod=98'1)", 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.247910+0000", 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.130538745000003, 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038978400000000001, 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247910+0000", 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247910+0000", 2024-09-13T10:35:14.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247910+0000", 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1999999999999999e-07 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247927+0000", 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7082e-05 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247963+0000", 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6012e-05 2024-09-13T10:35:14.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247971+0000", 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4460000000000004e-06 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248014+0000", 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3723000000000001e-05 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248027+0000", 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2558000000000001e-05 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248266+0000", 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000238827 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248299+0000", 2024-09-13T10:35:14.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3416e-05 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "pg_update_log_missing(2.d epoch 98/97 rep_tid 0 entries 98'1 (0'0) error 2:b1b22a0f:::image17:head by client.24508.0:142 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.254853+0000", 2024-09-13T10:35:14.405 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.123595651999999, 2024-09-13T10:35:14.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00056983399999999997, 2024-09-13T10:35:14.406 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.406 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.406 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.254853+0000", 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.254853+0000", 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.254853+0000", 2024-09-13T10:35:14.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.09e-07 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.254858+0000", 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7100000000000004e-06 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.254900+0000", 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1841000000000001e-05 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.254919+0000", 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8943000000000001e-05 2024-09-13T10:35:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.255032+0000", 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011328300000000001 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.255422+0000", 2024-09-13T10:35:14.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038994799999999999 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:143 2.d e98/97 2:b1b22a0f:::image17:head v 98'2, mlcod=0'0)", 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.256026+0000", 2024-09-13T10:35:14.412 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.122422370000002, 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00053607899999999996, 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256026+0000", 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256026+0000", 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256027+0000", 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9699999999999995e-07 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256032+0000", 2024-09-13T10:35:14.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3580000000000002e-06 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256059+0000", 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7183e-05 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256065+0000", 2024-09-13T10:35:14.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5400000000000003e-06 2024-09-13T10:35:14.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256123+0000", 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8121999999999997e-05 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256143+0000", 2024-09-13T10:35:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0222000000000001e-05 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256516+0000", 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000372913 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.256562+0000", 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6044000000000002e-05 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.420 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:145 2.d e98/97 2:b1b22a0f:::image17:head v 98'3, mlcod=98'1)", 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.257432+0000", 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.121016402000002, 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00074766100000000005, 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257432+0000", 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257432+0000", 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257433+0000", 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5960000000000001e-06 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257445+0000", 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1197e-05 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257508+0000", 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2939000000000002e-05 2024-09-13T10:35:14.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257520+0000", 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2109999999999999e-05 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257659+0000", 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000139367 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.257717+0000", 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7704999999999997e-05 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.258143+0000", 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00042661499999999998 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.258180+0000", 2024-09-13T10:35:14.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6131999999999999e-05 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:147 2.d e98/97 2:b1b22a0f:::image17:head v 98'4, mlcod=98'2)", 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.258971+0000", 2024-09-13T10:35:14.428 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 40.119477525999997, 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00041834899999999999, 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.258971+0000", 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.258971+0000", 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.258971+0000", 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0900000000000001e-07 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.258977+0000", 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3689999999999998e-06 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.259004+0000", 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7518e-05 2024-09-13T10:35:14.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.259010+0000", 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8309999999999997e-06 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.259051+0000", 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0411999999999999e-05 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.259069+0000", 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8326e-05 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.259354+0000", 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028515700000000002 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.259389+0000", 2024-09-13T10:35:14.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5027000000000002e-05 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "replica_scrub(pg: 1.d,from:0'0,to:0'0,epoch:122/121,start:1:b0000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-13T10:35:14.436 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:35:13.373736+0000", 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.0047123179999999, 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012947300000000001, 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373736+0000", 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373736+0000", 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373737+0000", 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8999999999999996e-07 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373742+0000", 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1839999999999998e-06 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373773+0000", 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1123e-05 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373781+0000", 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1159999999999999e-06 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373829+0000", 2024-09-13T10:35:14.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7988999999999997e-05 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373840+0000", 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0713999999999999e-05 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.373865+0000", 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5656999999999998e-05 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "rep_scrubmap(1.e e121 from shard 7)", 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:35:13.763002+0000", 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.61544640500000003, 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010813, 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:14.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763002+0000", 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763002+0000", 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:14.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763002+0000", 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4799999999999998e-07 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763006+0000", 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9809999999999997e-06 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763030+0000", 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3349e-05 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763035+0000", 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4349999999999996e-06 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:14.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763083+0000", 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8056999999999997e-05 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763091+0000", 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6969999999999993e-06 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:14.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:35:13.763110+0000", 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8963000000000001e-05 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:14.450 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:14.451 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:14.480 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-13T10:35:14.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pgp_num 21 2024-09-13T10:35:14.509 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.1 scrub starts 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.1 scrub ok 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1498255745' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: osdmap e122: 12 total, 12 up, 12 in 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.13 scrub starts 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.13 scrub ok 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.d scrub starts 2024-09-13T10:35:14.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.d scrub ok 2024-09-13T10:35:14.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.b scrub starts 2024-09-13T10:35:14.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.b scrub ok 2024-09-13T10:35:14.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.f scrub starts 2024-09-13T10:35:14.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.f scrub ok 2024-09-13T10:35:14.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.e scrub starts 2024-09-13T10:35:14.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:14 smithi060 ceph-mon[27375]: 1.e scrub ok 2024-09-13T10:35:14.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.1 scrub starts 2024-09-13T10:35:14.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.1 scrub ok 2024-09-13T10:35:14.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1498255745' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-13T10:35:14.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: osdmap e122: 12 total, 12 up, 12 in 2024-09-13T10:35:14.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.13 scrub starts 2024-09-13T10:35:14.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.13 scrub ok 2024-09-13T10:35:14.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.d scrub starts 2024-09-13T10:35:14.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.d scrub ok 2024-09-13T10:35:14.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.b scrub starts 2024-09-13T10:35:14.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.b scrub ok 2024-09-13T10:35:14.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.f scrub starts 2024-09-13T10:35:14.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.f scrub ok 2024-09-13T10:35:14.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.e scrub starts 2024-09-13T10:35:14.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:14 smithi173 ceph-mon[30864]: 1.e scrub ok 2024-09-13T10:35:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:15.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-13T10:35:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:15.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.10 scrub starts 2024-09-13T10:35:15.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.10 scrub ok 2024-09-13T10:35:15.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.6 scrub starts 2024-09-13T10:35:15.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.6 scrub ok 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.14 scrub starts 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.14 scrub ok 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.c scrub starts 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.c scrub ok 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: pgmap v353: 21 pgs: 4 peering, 4 unknown, 13 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.11 scrub starts 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: 1.11 scrub ok 2024-09-13T10:35:15.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:15 smithi060 ceph-mon[27375]: osdmap e123: 12 total, 12 up, 12 in 2024-09-13T10:35:15.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.10 scrub starts 2024-09-13T10:35:15.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.10 scrub ok 2024-09-13T10:35:15.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.6 scrub starts 2024-09-13T10:35:15.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.6 scrub ok 2024-09-13T10:35:15.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.14 scrub starts 2024-09-13T10:35:15.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.14 scrub ok 2024-09-13T10:35:15.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.c scrub starts 2024-09-13T10:35:15.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.c scrub ok 2024-09-13T10:35:15.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: pgmap v353: 21 pgs: 4 peering, 4 unknown, 13 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:15.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.11 scrub starts 2024-09-13T10:35:15.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: 1.11 scrub ok 2024-09-13T10:35:15.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:15 smithi173 ceph-mon[30864]: osdmap e123: 12 total, 12 up, 12 in 2024-09-13T10:35:15.588 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.10 scrub starts 2024-09-13T10:35:15.588 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.10 scrub ok 2024-09-13T10:35:15.588 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.6 scrub starts 2024-09-13T10:35:15.588 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.6 scrub ok 2024-09-13T10:35:15.588 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.14 scrub starts 2024-09-13T10:35:15.588 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.14 scrub ok 2024-09-13T10:35:15.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.c scrub starts 2024-09-13T10:35:15.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.c scrub ok 2024-09-13T10:35:15.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: pgmap v353: 21 pgs: 4 peering, 4 unknown, 13 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:15.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.11 scrub starts 2024-09-13T10:35:15.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: 1.11 scrub ok 2024-09-13T10:35:15.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:15 smithi138 ceph-mon[31033]: osdmap e123: 12 total, 12 up, 12 in 2024-09-13T10:35:15.940 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-13T10:35:15.940 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:16.173 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pgp_num to 21 2024-09-13T10:35:16.277 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:16 smithi138 ceph-mon[31033]: 1.12 scrub starts 2024-09-13T10:35:16.278 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:16 smithi138 ceph-mon[31033]: 1.12 scrub ok 2024-09-13T10:35:16.278 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:16 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/206990863' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"} : dispatch 2024-09-13T10:35:16.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:16 smithi060 ceph-mon[27375]: 1.12 scrub starts 2024-09-13T10:35:16.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:16 smithi060 ceph-mon[27375]: 1.12 scrub ok 2024-09-13T10:35:16.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:16 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/206990863' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"} : dispatch 2024-09-13T10:35:16.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:16 smithi173 ceph-mon[30864]: 1.12 scrub starts 2024-09-13T10:35:16.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:16 smithi173 ceph-mon[30864]: 1.12 scrub ok 2024-09-13T10:35:16.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:16 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/206990863' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"} : dispatch 2024-09-13T10:35:16.678 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-13T10:35:16.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T10:35:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:17.147 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:17.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:17 smithi138 ceph-mon[31033]: pgmap v355: 21 pgs: 3 peering, 1 unknown, 17 active+clean; 842 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:17.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:17 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/206990863' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-13T10:35:17.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:17 smithi138 ceph-mon[31033]: osdmap e124: 12 total, 12 up, 12 in 2024-09-13T10:35:17.388 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:17.388 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:17.389 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-13T10:35:17.389 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-13T10:35:17.389 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:17.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:17 smithi173 ceph-mon[30864]: pgmap v355: 21 pgs: 3 peering, 1 unknown, 17 active+clean; 842 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:17.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:17 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/206990863' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-13T10:35:17.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:17 smithi173 ceph-mon[30864]: osdmap e124: 12 total, 12 up, 12 in 2024-09-13T10:35:17.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:17 smithi060 ceph-mon[27375]: pgmap v355: 21 pgs: 3 peering, 1 unknown, 17 active+clean; 842 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:17.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:17 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/206990863' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-13T10:35:17.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:17 smithi060 ceph-mon[27375]: osdmap e124: 12 total, 12 up, 12 in 2024-09-13T10:35:17.639 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a: {} 2024-09-13T10:35:17.640 INFO:teuthology.orchestra.run.smithi060.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:35:17.654 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c: {} 2024-09-13T10:35:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:35:17.667 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b: {} 2024-09-13T10:35:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:35:18.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-13T10:35:18.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:18 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1910236699' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:35:18.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-13T10:35:18.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:18 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1910236699' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:35:18.369 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:18.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:18 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1910236699' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-13T10:35:18.574 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:18.856 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:18.856 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-13T10:35:18.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-13T10:35:18.856 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-13T10:35:18.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.857 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:2 2.2 2:424638ea:::image1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.857 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.548441+0000", 2024-09-13T10:35:18.857 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.304146803999998, 2024-09-13T10:35:18.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00119754, 2024-09-13T10:35:18.857 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.858 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.858 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.858 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.858 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.859 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 2 2024-09-13T10:35:18.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.859 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548441+0000", 2024-09-13T10:35:18.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548441+0000", 2024-09-13T10:35:18.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548443+0000", 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0379999999999998e-06 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548450+0000", 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1509999999999998e-06 2024-09-13T10:35:18.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548464+0000", 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3288999999999999e-05 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548475+0000", 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1575e-05 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548715+0000", 2024-09-13T10:35:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000239275 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548805+0000", 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0623000000000003e-05 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 1", 2024-09-13T10:35:18.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.548928+0000", 2024-09-13T10:35:18.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012228299999999999 2024-09-13T10:35:18.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.549384+0000", 2024-09-13T10:35:18.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00045697400000000002 2024-09-13T10:35:18.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.549584+0000", 2024-09-13T10:35:18.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019989899999999999 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.549615+0000", 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0630000000000003e-05 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.549639+0000", 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3802999999999999e-05 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.869 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:14 2.16 2:6bb855e4:::image3:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.716217+0000", 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.136370855000003, 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0011827949999999999, 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.871 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 14 2024-09-13T10:35:18.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.871 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716217+0000", 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716217+0000", 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716219+0000", 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3030000000000001e-06 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716224+0000", 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5990000000000001e-06 2024-09-13T10:35:18.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716239+0000", 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4778e-05 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716247+0000", 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3329999999999997e-06 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716371+0000", 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012362199999999999 2024-09-13T10:35:18.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.716455+0000", 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4110999999999995e-05 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717400+0000", 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000945049 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.877 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:15 2.16 2:6bb855e4:::image3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.878 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.717527+0000", 2024-09-13T10:35:18.878 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.135060639999999, 2024-09-13T10:35:18.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000944567, 2024-09-13T10:35:18.878 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.878 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 15 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.879 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717527+0000", 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717527+0000", 2024-09-13T10:35:18.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717528+0000", 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0550000000000001e-06 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717547+0000", 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8094e-05 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717561+0000", 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4090000000000001e-05 2024-09-13T10:35:18.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717567+0000", 2024-09-13T10:35:18.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0990000000000004e-06 2024-09-13T10:35:18.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717609+0000", 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2026e-05 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717663+0000", 2024-09-13T10:35:18.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4227000000000001e-05 2024-09-13T10:35:18.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.717754+0000", 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1188999999999997e-05 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718091+0000", 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033648599999999999 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718410+0000", 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031973599999999999 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718445+0000", 2024-09-13T10:35:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4604000000000003e-05 2024-09-13T10:35:18.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718472+0000", 2024-09-13T10:35:18.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6961000000000002e-05 2024-09-13T10:35:18.888 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:16 2.16 2:6bb855e4:::image3:head [write 1~1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.718572+0000", 2024-09-13T10:35:18.889 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.134016172000003, 2024-09-13T10:35:18.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00094484899999999999, 2024-09-13T10:35:18.890 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.890 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 16 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718572+0000", 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718572+0000", 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718573+0000", 2024-09-13T10:35:18.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1969999999999999e-06 2024-09-13T10:35:18.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718578+0000", 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9590000000000003e-06 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718590+0000", 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.163e-05 2024-09-13T10:35:18.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718595+0000", 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718634+0000", 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8522999999999997e-05 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718688+0000", 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3896e-05 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.718775+0000", 2024-09-13T10:35:18.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7143000000000002e-05 2024-09-13T10:35:18.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719178+0000", 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00040318 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719457+0000", 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027894600000000002 2024-09-13T10:35:18.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719489+0000", 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2308999999999998e-05 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719517+0000", 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7268e-05 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:17 2.16 2:6bb855e4:::image3:head [write 3~1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.719645+0000", 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.132943156000003, 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0038454349999999999, 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.901 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719645+0000", 2024-09-13T10:35:18.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719645+0000", 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719646+0000", 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.269e-06 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719653+0000", 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3099999999999997e-06 2024-09-13T10:35:18.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719665+0000", 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2251e-05 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719672+0000", 2024-09-13T10:35:18.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0720000000000002e-06 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719715+0000", 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3624000000000001e-05 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719762+0000", 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6909000000000001e-05 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719841+0000", 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8704999999999999e-05 2024-09-13T10:35:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.723262+0000", 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0034205199999999998 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.723430+0000", 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016848399999999999 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.723464+0000", 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3728e-05 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.723490+0000", 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6563e-05 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:21 2.16 2:6bb855e4:::image3:head [call rbd.sparse_copyup in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.912 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.725872+0000", 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.126715615999998, 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010872499999999999, 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 21 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725872+0000", 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725872+0000", 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725874+0000", 2024-09-13T10:35:18.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.147e-06 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725879+0000", 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.356e-06 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725894+0000", 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4958e-05 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725900+0000", 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-13T10:35:18.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.725958+0000", 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7982000000000002e-05 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.726036+0000", 2024-09-13T10:35:18.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.784e-05 2024-09-13T10:35:18.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.726225+0000", 2024-09-13T10:35:18.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018846800000000001 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.726601+0000", 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00037607899999999997 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.726885+0000", 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028424899999999998 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.726927+0000", 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2058000000000003e-05 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.726960+0000", 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2691999999999998e-05 2024-09-13T10:35:18.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:24 2.16 2:6bb855e4:::image3:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.728837+0000", 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.123750932999997, 2024-09-13T10:35:18.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.39392582599999998, 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 24 2024-09-13T10:35:18.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728837+0000", 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728837+0000", 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728838+0000", 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0210000000000001e-06 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728843+0000", 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4819999999999996e-06 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728856+0000", 2024-09-13T10:35:18.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.296e-05 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728861+0000", 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7869999999999998e-06 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728924+0000", 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2655000000000002e-05 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.728998+0000", 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4049999999999997e-05 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122763+0000", 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.39376487100000002 2024-09-13T10:35:18.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:25 2.16 2:6bb855e4:::image3:head [call rbd.sparse_copyup in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.122944+0000", 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 44.729643830999997, 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00117034, 2024-09-13T10:35:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 25 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.932 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122944+0000", 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122944+0000", 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122946+0000", 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4580000000000001e-06 2024-09-13T10:35:18.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122968+0000", 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2631000000000001e-05 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122984+0000", 2024-09-13T10:35:18.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5366000000000002e-05 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122991+0000", 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4220000000000003e-06 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.123041+0000", 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0037999999999998e-05 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.123111+0000", 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9356999999999993e-05 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.123278+0000", 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016708099999999999 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.123787+0000", 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00050947799999999995 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124043+0000", 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000255467 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124081+0000", 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8056000000000003e-05 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124115+0000", 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3986000000000003e-05 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:26 2.16 2:6bb855e4:::image3:head [read 0~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.124300+0000", 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 44.728288055, 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0032348239999999999, 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:18.942 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 26 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124300+0000", 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124300+0000", 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124301+0000", 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124313+0000", 2024-09-13T10:35:18.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2401999999999999e-05 2024-09-13T10:35:18.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124325+0000", 2024-09-13T10:35:18.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1078e-05 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124332+0000", 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5449999999999998e-06 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124374+0000", 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1501999999999997e-05 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.124424+0000", 2024-09-13T10:35:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0754000000000002e-05 2024-09-13T10:35:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.127535+0000", 2024-09-13T10:35:18.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0031105320000000001 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:151 2.16 2:6ba089ba:::image18:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.261207+0000", 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 44.591381404000003, 2024-09-13T10:35:18.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.97316037099999997, 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.951 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 151 2024-09-13T10:35:18.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.952 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261207+0000", 2024-09-13T10:35:18.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261207+0000", 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261207+0000", 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3200000000000005e-07 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261212+0000", 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.25e-06 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261221+0000", 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8910000000000007e-06 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261226+0000", 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9579999999999998e-06 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261275+0000", 2024-09-13T10:35:18.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8662999999999998e-05 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261350+0000", 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4436000000000004e-05 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261570+0000", 2024-09-13T10:35:18.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00022055499999999999 2024-09-13T10:35:18.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.261924+0000", 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000353629 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234258+0000", 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.97233407800000005 2024-09-13T10:35:18.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234322+0000", 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4480000000000006e-05 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234367+0000", 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4799000000000003e-05 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.961 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop_reply(client.24508.0:151 2.16 e98/97 ondisk, result = 0)", 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.262033+0000", 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 44.590555246000001, 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.97234384100000004, 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.262033+0000", 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.262033+0000", 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.262033+0000", 2024-09-13T10:35:18.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0399999999999996e-07 2024-09-13T10:35:18.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.262037+0000", 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5999999999999998e-06 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234065+0000", 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.97202849899999999 2024-09-13T10:35:18.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234106+0000", 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0355999999999998e-05 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234234+0000", 2024-09-13T10:35:18.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012812899999999999 2024-09-13T10:35:18.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234249+0000", 2024-09-13T10:35:18.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4917e-05 2024-09-13T10:35:18.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.234377+0000", 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012783599999999999 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.969 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:156 2.16 2:6ba089ba:::image18:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.248315+0000", 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.604273233999997, 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001133996, 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 156 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248315+0000", 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248315+0000", 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248316+0000", 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1969999999999999e-06 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248323+0000", 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9029999999999998e-06 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248337+0000", 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4528e-05 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248344+0000", 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3500000000000002e-06 2024-09-13T10:35:18.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248402+0000", 2024-09-13T10:35:18.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8082999999999999e-05 2024-09-13T10:35:18.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.248463+0000", 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0676e-05 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.249449+0000", 2024-09-13T10:35:18.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00098625899999999996 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:158 2.16 2:6ba089ba:::image18:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.250186+0000", 2024-09-13T10:35:18.977 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.602402204000001, 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00098534800000000004, 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.978 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 158 2024-09-13T10:35:18.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.979 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250186+0000", 2024-09-13T10:35:18.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250186+0000", 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250187+0000", 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.128e-06 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250194+0000", 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410000000000003e-06 2024-09-13T10:35:18.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250205+0000", 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1430999999999999e-05 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250211+0000", 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4060000000000004e-06 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250257+0000", 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6706999999999999e-05 2024-09-13T10:35:18.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.250309+0000", 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1993999999999997e-05 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251171+0000", 2024-09-13T10:35:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00086194099999999997 2024-09-13T10:35:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:159 2.16 2:6ba089ba:::image18:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.251452+0000", 2024-09-13T10:35:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.601136461999999, 2024-09-13T10:35:18.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0011789470000000001, 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 159 2024-09-13T10:35:18.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251452+0000", 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251452+0000", 2024-09-13T10:35:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251452+0000", 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.23e-07 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251458+0000", 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7769999999999999e-06 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251471+0000", 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2323e-05 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251476+0000", 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8409999999999996e-06 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251518+0000", 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1637999999999997e-05 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251572+0000", 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3995000000000001e-05 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:18.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.251826+0000", 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025374600000000001 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.252196+0000", 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00037045999999999997 2024-09-13T10:35:18.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.252535+0000", 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033918499999999999 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:18.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.252569+0000", 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3111000000000002e-05 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.252631+0000", 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2048e-05 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.996 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:161 2.16 2:6ba089ba:::image18:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.253333+0000", 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.599255251999999, 2024-09-13T10:35:18.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010059999999999999, 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 161 2024-09-13T10:35:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:18.999 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:18.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:18.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:18.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253333+0000", 2024-09-13T10:35:18.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:18.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253333+0000", 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:19.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253334+0000", 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9299999999999997e-07 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253338+0000", 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4959999999999998e-06 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253349+0000", 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0497e-05 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253355+0000", 2024-09-13T10:35:19.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.489e-06 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253395+0000", 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0074999999999998e-05 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253450+0000", 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4688000000000002e-05 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253591+0000", 2024-09-13T10:35:19.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000140949 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.253895+0000", 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000303916 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:19.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.254266+0000", 2024-09-13T10:35:19.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00037097299999999997 2024-09-13T10:35:19.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.254297+0000", 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1367e-05 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.254339+0000", 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1757e-05 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:164 2.16 2:6ba089ba:::image18:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:19.010 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.256249+0000", 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.596339004999997, 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00098417399999999999, 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:19.011 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:19.012 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 164 2024-09-13T10:35:19.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.012 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:19.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:19.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256249+0000", 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256249+0000", 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256250+0000", 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.046e-06 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256256+0000", 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5629999999999999e-06 2024-09-13T10:35:19.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256269+0000", 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3644e-05 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256275+0000", 2024-09-13T10:35:19.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8340000000000004e-06 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256347+0000", 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1429999999999999e-05 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256399+0000", 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1986e-05 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256532+0000", 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013316399999999999 2024-09-13T10:35:19.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.256858+0000", 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032601400000000001 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.257161+0000", 2024-09-13T10:35:19.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00030339000000000001 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.257191+0000", 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0113000000000001e-05 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.257233+0000", 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1990000000000003e-05 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:169 2.16 2:6ba089ba:::image18:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:19.020 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.260758+0000", 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.591830158999997, 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010432659999999999, 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:19.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 169 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260758+0000", 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260758+0000", 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260759+0000", 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.082e-06 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260764+0000", 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0039999999999999e-06 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260778+0000", 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.414e-05 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260785+0000", 2024-09-13T10:35:19.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4350000000000002e-06 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260856+0000", 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1403999999999996e-05 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.260909+0000", 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3009e-05 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.261128+0000", 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021910300000000001 2024-09-13T10:35:19.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.261472+0000", 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00034367600000000001 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.261730+0000", 2024-09-13T10:35:19.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025835700000000002 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.261761+0000", 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0534999999999998e-05 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.261801+0000", 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0521e-05 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:171 2.16 2:6ba089ba:::image18:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.262672+0000", 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.589916299000002, 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001018478, 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:19.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 171 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262672+0000", 2024-09-13T10:35:19.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262672+0000", 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262673+0000", 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.082e-06 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262686+0000", 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2916e-05 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262700+0000", 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3926e-05 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262707+0000", 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5259999999999996e-06 2024-09-13T10:35:19.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262775+0000", 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7607000000000005e-05 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:19.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262826+0000", 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1582999999999997e-05 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.262975+0000", 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014901500000000001 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.263295+0000", 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000319888 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.263618+0000", 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032291900000000002 2024-09-13T10:35:19.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.263650+0000", 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1368999999999997e-05 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.263690+0000", 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0646999999999999e-05 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:173 2.16 2:6ba089ba:::image18:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:19.040 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.264297+0000", 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.588291036000001, 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00094486099999999996, 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:19.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 173 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264297+0000", 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264297+0000", 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:19.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264298+0000", 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9999999999999996e-07 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264303+0000", 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2510000000000002e-06 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264313+0000", 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0125e-05 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264318+0000", 2024-09-13T10:35:19.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2480000000000004e-06 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264359+0000", 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0300999999999999e-05 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264407+0000", 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8387000000000003e-05 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264586+0000", 2024-09-13T10:35:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017873499999999999 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.264879+0000", 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029357299999999999 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265172+0000", 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029238799999999998 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265202+0000", 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0556999999999999e-05 2024-09-13T10:35:19.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265242+0000", 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9496000000000003e-05 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:174 2.16 2:6ba089ba:::image18:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.265448+0000", 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.587139856, 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00094960900000000002, 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:19.051 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 174 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:19.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265448+0000", 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265448+0000", 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:19.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265449+0000", 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4300000000000002e-07 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265454+0000", 2024-09-13T10:35:19.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5229999999999997e-06 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265465+0000", 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1163e-05 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265470+0000", 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2109999999999997e-06 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265509+0000", 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9419000000000003e-05 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265558+0000", 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8668999999999998e-05 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.265730+0000", 2024-09-13T10:35:19.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017229800000000001 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.266033+0000", 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00030257999999999999 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.266327+0000", 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029388899999999998 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.266358+0000", 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1136999999999997e-05 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:19.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.266398+0000", 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9876999999999998e-05 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:19.060 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:19.061 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:19.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:19 smithi060 ceph-mon[27375]: pgmap v357: 21 pgs: 1 peering, 20 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:19.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:19 smithi173 ceph-mon[30864]: pgmap v357: 21 pgs: 1 peering, 20 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:19.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:19 smithi138 ceph-mon[31033]: pgmap v357: 21 pgs: 1 peering, 20 active+clean; 603 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:19.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-13T10:35:19.975 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T10:35:20.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:20 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3831282538' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-13T10:35:20.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:20 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3831282538' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-13T10:35:20.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:20 smithi138 ceph-mon[31033]: osdmap e125: 12 total, 12 up, 12 in 2024-09-13T10:35:20.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:20 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3831282538' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-13T10:35:20.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:20 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3831282538' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-13T10:35:20.365 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:20 smithi060 ceph-mon[27375]: osdmap e125: 12 total, 12 up, 12 in 2024-09-13T10:35:20.415 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:20.415 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:20.415 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-13T10:35:20.416 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:20.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:20 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3831282538' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-13T10:35:20.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:20 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3831282538' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-13T10:35:20.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:20 smithi173 ceph-mon[30864]: osdmap e125: 12 total, 12 up, 12 in 2024-09-13T10:35:20.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 0 1 2024-09-13T10:35:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:21.101 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-13T10:35:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:21.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:21 smithi138 ceph-mon[31033]: pgmap v358: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:21.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:21 smithi060 ceph-mon[27375]: pgmap v358: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:21.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:21 smithi173 ceph-mon[30864]: pgmap v358: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:21.830 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:21.830 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:21.830 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-13T10:35:21.830 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-13T10:35:21.831 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T10:35:22.482 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:22 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3987165493' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-13T10:35:22.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:22 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3987165493' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-13T10:35:22.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:22 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3987165493' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-13T10:35:22.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-13T10:35:22.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-13T10:35:22.876 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:23.102 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:23.281 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:23 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:23.281 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:23 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:23.281 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:23 smithi138 ceph-mon[31033]: pgmap v360: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:23.281 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:23 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3987165493' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-13T10:35:23.282 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:23 smithi138 ceph-mon[31033]: osdmap e126: 12 total, 12 up, 12 in 2024-09-13T10:35:23.383 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:23.385 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDPGPush(1.0 48/47 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'91, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'91, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:31:18.733741+0000", 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 244.644009216, 2024-09-13T10:35:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00095478999999999996, 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.733741+0000", 2024-09-13T10:35:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.733741+0000", 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.733742+0000", 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.734059+0000", 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031676499999999999 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.734099+0000", 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9759000000000003e-05 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.734122+0000", 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2942000000000002e-05 2024-09-13T10:35:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.734185+0000", 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3676000000000005e-05 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.734194+0000", 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2550000000000003e-06 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:18.734696+0000", 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00050173000000000004 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDPGPull(1.0 e54/53 cost 1049576)", 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:31:44.650439+0000", 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 218.727312109, 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0027416540000000001, 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650439+0000", 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650439+0000", 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650439+0000", 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1900000000000002e-07 2024-09-13T10:35:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650447+0000", 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9820000000000008e-06 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650470+0000", 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2575e-05 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650476+0000", 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2029999999999997e-06 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650519+0000", 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2960000000000002e-05 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653180+0000", 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002661315 2024-09-13T10:35:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.24508.0:17 2.16 e98/97 2:6bb855e4:::image3:head v 98'4, mlcod=98'2)", 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.719909+0000", 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.657841585, 2024-09-13T10:35:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0033738319999999998, 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719909+0000", 2024-09-13T10:35:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719909+0000", 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719910+0000", 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3399999999999998e-07 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719922+0000", 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.188e-05 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719952+0000", 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950999999999998e-05 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.719958+0000", 2024-09-13T10:35:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4389999999999997e-06 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.720004+0000", 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6108e-05 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.720018+0000", 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.393e-05 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.723247+0000", 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003228365 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.723283+0000", 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6325000000000002e-05 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "pg_update_log_missing(2.16 epoch 98/97 rep_tid 8 entries 98'8 (0'0) error 2:6bb855e4:::image3:head by client.24508.0:24 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 98'6)", 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:33.729259+0000", 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.648491810000003, 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.393242325, 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.729259+0000", 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.729259+0000", 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.729260+0000", 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1299999999999999e-07 2024-09-13T10:35:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:33.729270+0000", 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.042e-05 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.121874+0000", 2024-09-13T10:35:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.39260442000000001 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.121901+0000", 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6359e-05 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122091+0000", 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018996900000000001 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.122501+0000", 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00041034399999999999 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:57 2.7 2:e5291ff9:::image6:head [call rbd.dir_state_set in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.173269+0000", 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.204481350000002, 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0011995549999999999, 2024-09-13T10:35:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 57 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173269+0000", 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173269+0000", 2024-09-13T10:35:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173270+0000", 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2800000000000005e-07 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173275+0000", 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1889999999999998e-06 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173284+0000", 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7539999999999995e-06 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173289+0000", 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4499999999999997e-06 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173323+0000", 2024-09-13T10:35:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3954000000000001e-05 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.173384+0000", 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1468000000000002e-05 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.174469+0000", 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010848120000000001 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:60 2.7 2:e5291ff9:::image6:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.175026+0000", 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.202724764000003, 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00085847600000000005, 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 60 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175026+0000", 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175026+0000", 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175027+0000", 2024-09-13T10:35:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7300000000000005e-07 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175030+0000", 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.743e-06 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175039+0000", 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9390000000000003e-06 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175044+0000", 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5129999999999998e-06 2024-09-13T10:35:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175079+0000", 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4904000000000003e-05 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175119+0000", 2024-09-13T10:35:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0311000000000001e-05 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175884+0000", 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00076519299999999995 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:61 2.7 2:e5291ff9:::image6:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.175982+0000", 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.201768489999999, 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010949309999999999, 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 61 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175982+0000", 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175982+0000", 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175983+0000", 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5499999999999996e-07 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175987+0000", 2024-09-13T10:35:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8850000000000001e-06 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.175996+0000", 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0929999999999992e-06 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.176001+0000", 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.493e-06 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.176041+0000", 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0917999999999997e-05 2024-09-13T10:35:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.176089+0000", 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7960000000000002e-05 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.177077+0000", 2024-09-13T10:35:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00098762700000000004 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:136 2.11 2:8dfbcd02:::image16:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.239770+0000", 2024-09-13T10:35:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.137980820999999, 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010670510000000001, 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 136 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239770+0000", 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239770+0000", 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239771+0000", 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6300000000000004e-07 2024-09-13T10:35:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239775+0000", 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-06 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239783+0000", 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6810000000000004e-06 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239798+0000", 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4357e-05 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239838+0000", 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0160999999999997e-05 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239937+0000", 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9226000000000002e-05 2024-09-13T10:35:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240082+0000", 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014517800000000001 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240386+0000", 2024-09-13T10:35:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00030428399999999999 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240782+0000", 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000395375 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240809+0000", 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7018000000000001e-05 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.240837+0000", 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8017999999999998e-05 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:138 2.11 2:8dfbcd02:::image16:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+peerstat_old+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.246220+0000", 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.131530607999998, 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001205708, 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 138 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246220+0000", 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246220+0000", 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246222+0000", 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.823e-06 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246230+0000", 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4540000000000007e-06 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246252+0000", 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1231999999999999e-05 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246266+0000", 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.469e-05 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246421+0000", 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00015489 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246547+0000", 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012630700000000001 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.246713+0000", 2024-09-13T10:35:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000165635 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247046+0000", 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033283500000000002 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247375+0000", 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032921300000000002 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247404+0000", 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8799e-05 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247426+0000", 2024-09-13T10:35:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1829999999999999e-05 2024-09-13T10:35:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:139 2.11 2:8dfbcd02:::image16:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.247600+0000", 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.130150213, 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00088135500000000001, 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 139 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247600+0000", 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247600+0000", 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247601+0000", 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0699999999999996e-07 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247607+0000", 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4990000000000002e-06 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247618+0000", 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1127e-05 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247623+0000", 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1410000000000004e-06 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247665+0000", 2024-09-13T10:35:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2274000000000002e-05 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247714+0000", 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8943000000000002e-05 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.247839+0000", 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000124282 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248240+0000", 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00040184999999999998 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248426+0000", 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000185114 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248456+0000", 2024-09-13T10:35:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0510000000000001e-05 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.248482+0000", 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5707999999999998e-05 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:176 2.17 2:e9c6360c:::image20:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.267089+0000", 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.110661274999998, 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0011910379999999999, 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 176 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267089+0000", 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267089+0000", 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267090+0000", 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267095+0000", 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6650000000000002e-06 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267104+0000", 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7220000000000005e-06 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267108+0000", 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6630000000000001e-06 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267145+0000", 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6684999999999999e-05 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267206+0000", 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1552999999999998e-05 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 10", 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267336+0000", 2024-09-13T10:35:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000130044 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.267645+0000", 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000308616 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268204+0000", 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00055942399999999999 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268237+0000", 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2103000000000001e-05 2024-09-13T10:35:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268280+0000", 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3807000000000003e-05 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:178 2.17 2:e9c6360c:::image20:head [call rbd.snapshot_set_limit in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.268680+0000", 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.109071131999997, 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001007616, 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 178 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268680+0000", 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268680+0000", 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268680+0000", 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9800000000000003e-07 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268683+0000", 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.83e-06 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268691+0000", 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.089e-06 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268695+0000", 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2270000000000004e-06 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268732+0000", 2024-09-13T10:35:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6235999999999997e-05 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268777+0000", 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5207000000000003e-05 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 10", 2024-09-13T10:35:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.268935+0000", 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00015826699999999999 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.269213+0000", 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027830699999999999 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.269632+0000", 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00041893900000000001 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.269663+0000", 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0264999999999999e-05 2024-09-13T10:35:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.269687+0000", 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4550999999999999e-05 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:180 2.17 2:e9c6360c:::image20:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.270106+0000", 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.107644694000001, 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00097852799999999999, 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 180 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270106+0000", 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270106+0000", 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270107+0000", 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3200000000000004e-07 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270110+0000", 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270117+0000", 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6939999999999995e-06 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270122+0000", 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3390000000000003e-06 2024-09-13T10:35:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270156+0000", 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4121000000000001e-05 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270199+0000", 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3016999999999998e-05 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 10", 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270393+0000", 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019436200000000001 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.270726+0000", 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000332455 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271004+0000", 2024-09-13T10:35:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027788599999999998 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271042+0000", 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7988000000000003e-05 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271084+0000", 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2917000000000003e-05 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:181 2.17 2:e9c6360c:::image20:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.271161+0000", 2024-09-13T10:35:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.106589722000002, 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00087787, 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 181 2024-09-13T10:35:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271161+0000", 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271161+0000", 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271162+0000", 2024-09-13T10:35:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7599999999999996e-07 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271166+0000", 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000003e-06 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271176+0000", 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6190000000000006e-06 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271180+0000", 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6940000000000003e-06 2024-09-13T10:35:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271219+0000", 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8788999999999997e-05 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271267+0000", 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8180000000000003e-05 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 10", 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271460+0000", 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019315200000000001 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271758+0000", 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029776500000000002 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.271985+0000", 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00022711399999999999 2024-09-13T10:35:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272015+0000", 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9652999999999999e-05 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272039+0000", 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3828e-05 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:182 2.17 2:e9c6360c:::image20:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.272122+0000", 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.105628191999998, 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00083312399999999997, 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 182 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272122+0000", 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272122+0000", 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272123+0000", 2024-09-13T10:35:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.47e-07 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272130+0000", 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1920000000000002e-06 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272139+0000", 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3270000000000007e-06 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272148+0000", 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9779999999999994e-06 2024-09-13T10:35:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272192+0000", 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3662000000000003e-05 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272240+0000", 2024-09-13T10:35:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8599e-05 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.272956+0000", 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00071541900000000002 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:183 2.17 2:e9c6360c:::image20:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.273060+0000", 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.104690560999998, 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00084631400000000003, 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 183 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273060+0000", 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273060+0000", 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273061+0000", 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5499999999999997e-07 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273066+0000", 2024-09-13T10:35:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0350000000000001e-06 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273074+0000", 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2679999999999994e-06 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273079+0000", 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5159999999999996e-06 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273116+0000", 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7812999999999999e-05 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273165+0000", 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.884e-05 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 10", 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273339+0000", 2024-09-13T10:35:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017336599999999999 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273629+0000", 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029002299999999998 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273853+0000", 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00022411600000000001 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273883+0000", 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0006999999999999e-05 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273906+0000", 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3575e-05 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:184 2.17 2:e9c6360c:::image20:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.273990+0000", 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.103760372000004, 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089976200000000002, 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 184 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273990+0000", 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273990+0000", 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273991+0000", 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3900000000000004e-07 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.273994+0000", 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2720000000000002e-06 2024-09-13T10:35:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274003+0000", 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2560000000000002e-06 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274007+0000", 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2849999999999996e-06 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274044+0000", 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6739999999999997e-05 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274088+0000", 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4926999999999999e-05 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 10", 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274252+0000", 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000163838 2024-09-13T10:35:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274493+0000", 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024107299999999999 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274822+0000", 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032811500000000003 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274852+0000", 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0505e-05 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.274890+0000", 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8012000000000001e-05 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:186 2.14 2:2ea5a297:::image21:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.276639+0000", 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.101112118000003, 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001189358, 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 186 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276639+0000", 2024-09-13T10:35:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276639+0000", 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276639+0000", 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.78e-07 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276645+0000", 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.135e-06 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276657+0000", 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2860000000000001e-05 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276665+0000", 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1520000000000003e-06 2024-09-13T10:35:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276726+0000", 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0971999999999998e-05 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276797+0000", 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276937+0000", 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013913900000000001 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.277305+0000", 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00036800899999999998 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.277768+0000", 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00046329999999999999 2024-09-13T10:35:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.277801+0000", 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2889999999999999e-05 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.277828+0000", 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7161999999999998e-05 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:188 2.14 2:2ea5a297:::image21:head [call rbd.set_parent in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.278438+0000", 2024-09-13T10:35:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.099313062, 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010502759999999999, 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 188 2024-09-13T10:35:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278438+0000", 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278438+0000", 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278438+0000", 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5199999999999996e-07 2024-09-13T10:35:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278442+0000", 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6720000000000002e-06 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278452+0000", 2024-09-13T10:35:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5750000000000007e-06 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278456+0000", 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6700000000000002e-06 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278524+0000", 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7792000000000004e-05 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.278603+0000", 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8564000000000002e-05 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279488+0000", 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00088525099999999998 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:189 2.14 2:2ea5a297:::image21:head [call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.279632+0000", 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 48.098118990000003, 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00097043200000000002, 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 189 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279632+0000", 2024-09-13T10:35:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279632+0000", 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279633+0000", 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0750000000000001e-06 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279638+0000", 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.78e-06 2024-09-13T10:35:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279668+0000", 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0171999999999998e-05 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279674+0000", 2024-09-13T10:35:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8479999999999997e-06 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279741+0000", 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7216000000000002e-05 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.279820+0000", 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9024999999999996e-05 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.280602+0000", 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00078231600000000002 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:23.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:23 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:23.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:23 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:23.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:23 smithi173 ceph-mon[30864]: pgmap v360: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:23.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:23 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3987165493' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-13T10:35:23.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:23 smithi173 ceph-mon[30864]: osdmap e126: 12 total, 12 up, 12 in 2024-09-13T10:35:23.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:23 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:23.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:23 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:23.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:23 smithi060 ceph-mon[27375]: pgmap v360: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:23.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:23 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3987165493' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-13T10:35:23.562 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:23 smithi060 ceph-mon[27375]: osdmap e126: 12 total, 12 up, 12 in 2024-09-13T10:35:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T10:35:24.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:24 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3643619644' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-13T10:35:24.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-13T10:35:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:24 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3643619644' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-13T10:35:24.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:24 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3643619644' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-13T10:35:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:24.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 1 1 2024-09-13T10:35:25.167 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:25.168 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:25.168 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-13T10:35:25.168 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:25.245 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:25.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: pgmap v362: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:25.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3643619644' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-13T10:35:25.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: osdmap e127: 12 total, 12 up, 12 in 2024-09-13T10:35:25.366 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:25.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:25.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:25.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:25 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:35:25.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: pgmap v362: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:25.416 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3643619644' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-13T10:35:25.416 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: osdmap e127: 12 total, 12 up, 12 in 2024-09-13T10:35:25.416 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:25.416 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:25.416 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:25.416 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:25 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:35:25.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: pgmap v362: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:25.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3643619644' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-13T10:35:25.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: osdmap e127: 12 total, 12 up, 12 in 2024-09-13T10:35:25.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:25.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:25.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:25.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:25 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' 2024-09-13T10:35:25.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-13T10:35:26.166 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:26.212 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T10:35:26.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:26 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1511349604' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-13T10:35:26.569 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:26 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1511349604' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-13T10:35:26.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:26 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1511349604' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-13T10:35:26.668 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-13T10:35:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-13T10:35:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-13T10:35:26.669 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:26.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-13T10:35:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:27.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:27 smithi138 ceph-mon[31033]: pgmap v364: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:27.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:27 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1511349604' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-13T10:35:27.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:27 smithi138 ceph-mon[31033]: osdmap e128: 12 total, 12 up, 12 in 2024-09-13T10:35:27.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-13T10:35:27.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:27 smithi060 ceph-mon[27375]: pgmap v364: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:27.468 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:27 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1511349604' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-13T10:35:27.468 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:27 smithi060 ceph-mon[27375]: osdmap e128: 12 total, 12 up, 12 in 2024-09-13T10:35:27.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:27 smithi173 ceph-mon[30864]: pgmap v364: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:27.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:27 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1511349604' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-13T10:35:27.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:27 smithi173 ceph-mon[30864]: osdmap e128: 12 total, 12 up, 12 in 2024-09-13T10:35:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T10:35:28.218 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T10:35:28.278 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDPGPush(1.0 54/53 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'91, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'91, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:31:44.649281+0000", 2024-09-13T10:35:28.279 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 223.54395871400001, 2024-09-13T10:35:28.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00078917699999999996, 2024-09-13T10:35:28.280 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.280 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.280 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:28.280 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649281+0000", 2024-09-13T10:35:28.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649281+0000", 2024-09-13T10:35:28.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649282+0000", 2024-09-13T10:35:28.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9400000000000005e-07 2024-09-13T10:35:28.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649514+0000", 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000232511 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649547+0000", 2024-09-13T10:35:28.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2577999999999999e-05 2024-09-13T10:35:28.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649561+0000", 2024-09-13T10:35:28.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3698e-05 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649638+0000", 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7155000000000002e-05 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.649647+0000", 2024-09-13T10:35:28.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7650000000000005e-06 2024-09-13T10:35:28.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.650070+0000", 2024-09-13T10:35:28.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00042287599999999998 2024-09-13T10:35:28.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.288 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDPGPush(1.0 54/53 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'92, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'92, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:31:44.653220+0000", 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 223.54001982, 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00068968199999999995, 2024-09-13T10:35:28.289 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653220+0000", 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653220+0000", 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653221+0000", 2024-09-13T10:35:28.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8100000000000002e-07 2024-09-13T10:35:28.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653429+0000", 2024-09-13T10:35:28.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020805899999999999 2024-09-13T10:35:28.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653462+0000", 2024-09-13T10:35:28.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3599999999999997e-05 2024-09-13T10:35:28.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653473+0000", 2024-09-13T10:35:28.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0743e-05 2024-09-13T10:35:28.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653543+0000", 2024-09-13T10:35:28.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0279999999999998e-05 2024-09-13T10:35:28.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653552+0000", 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.157e-06 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:44.653910+0000", 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00035816199999999998 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.297 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDScrubReserve(1.0,GRANT,e:57,from:0,reservation_nonce:0)", 2024-09-13T10:35:28.298 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:31:58.885826+0000", 2024-09-13T10:35:28.298 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 209.30741387800001, 2024-09-13T10:35:28.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00036289599999999998, 2024-09-13T10:35:28.298 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.298 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:28.299 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885826+0000", 2024-09-13T10:35:28.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885826+0000", 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885827+0000", 2024-09-13T10:35:28.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0219999999999999e-06 2024-09-13T10:35:28.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885832+0000", 2024-09-13T10:35:28.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5549999999999996e-06 2024-09-13T10:35:28.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885861+0000", 2024-09-13T10:35:28.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9677999999999999e-05 2024-09-13T10:35:28.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885872+0000", 2024-09-13T10:35:28.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0506000000000001e-05 2024-09-13T10:35:28.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.885991+0000", 2024-09-13T10:35:28.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000118799 2024-09-13T10:35:28.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.886009+0000", 2024-09-13T10:35:28.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.861e-05 2024-09-13T10:35:28.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:31:58.886189+0000", 2024-09-13T10:35:28.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017972600000000001 2024-09-13T10:35:28.309 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.309 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.309 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.310 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDPGPushReply(1.0 63/62 [PushReplyOp(1:30306672:devicehealth::main.db.0000000000000000:head)])", 2024-09-13T10:35:28.310 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:32:09.572476+0000", 2024-09-13T10:35:28.310 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 198.620763861, 2024-09-13T10:35:28.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033085999999999999, 2024-09-13T10:35:28.311 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.311 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.311 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-13T10:35:28.311 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572476+0000", 2024-09-13T10:35:28.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572476+0000", 2024-09-13T10:35:28.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572476+0000", 2024-09-13T10:35:28.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.17e-07 2024-09-13T10:35:28.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572478+0000", 2024-09-13T10:35:28.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3460000000000001e-06 2024-09-13T10:35:28.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572526+0000", 2024-09-13T10:35:28.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7367999999999998e-05 2024-09-13T10:35:28.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572532+0000", 2024-09-13T10:35:28.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7780000000000004e-06 2024-09-13T10:35:28.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572595+0000", 2024-09-13T10:35:28.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2928000000000005e-05 2024-09-13T10:35:28.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:32:09.572807+0000", 2024-09-13T10:35:28.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021232299999999999 2024-09-13T10:35:28.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.322 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "MOSDPGPushReply(1.0 86/85 [PushReplyOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head)])", 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:33:33.768179+0000", 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.425061184, 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033775500000000002, 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.323 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768179+0000", 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768179+0000", 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768179+0000", 2024-09-13T10:35:28.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7800000000000001e-07 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768186+0000", 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7139999999999996e-06 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768233+0000", 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7027999999999998e-05 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768241+0000", 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8909999999999995e-06 2024-09-13T10:35:28.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768297+0000", 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6320999999999998e-05 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:33:33.768516+0000", 2024-09-13T10:35:28.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021902299999999999 2024-09-13T10:35:28.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.330 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:41 2.8 2:120d515b:::image5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.330 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.149167+0000", 2024-09-13T10:35:28.330 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.044072921999998, 2024-09-13T10:35:28.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001189601, 2024-09-13T10:35:28.330 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 41 2024-09-13T10:35:28.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.332 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149167+0000", 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149167+0000", 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149169+0000", 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2740000000000002e-06 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149180+0000", 2024-09-13T10:35:28.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0530000000000001e-05 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149200+0000", 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0525000000000001e-05 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149215+0000", 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4824000000000001e-05 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149300+0000", 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5329999999999998e-05 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149385+0000", 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4759999999999995e-05 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149474+0000", 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8721000000000007e-05 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.149825+0000", 2024-09-13T10:35:28.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000351357 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150295+0000", 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00046931300000000001 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150327+0000", 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2666999999999997e-05 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150357+0000", 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9300000000000001e-05 2024-09-13T10:35:28.340 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:43 2.8 2:120d515b:::image5:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.150829+0000", 2024-09-13T10:35:28.341 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.042411227000002, 2024-09-13T10:35:28.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00090214699999999995, 2024-09-13T10:35:28.342 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.342 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.342 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:28.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.343 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.343 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.343 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 43 2024-09-13T10:35:28.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.343 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150829+0000", 2024-09-13T10:35:28.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150829+0000", 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150830+0000", 2024-09-13T10:35:28.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150834+0000", 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3939999999999998e-06 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150842+0000", 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2549999999999997e-06 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150847+0000", 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5979999999999999e-06 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150887+0000", 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0049999999999998e-05 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.150949+0000", 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1884999999999994e-05 2024-09-13T10:35:28.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151731+0000", 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00078204399999999997 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:44 2.8 2:144d5af5:::parent_id:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.151852+0000", 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.041387651000001, 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00099194699999999997, 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 44 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151852+0000", 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151852+0000", 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151853+0000", 2024-09-13T10:35:28.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9999999999999995e-07 2024-09-13T10:35:28.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151858+0000", 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7589999999999997e-06 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151873+0000", 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4629e-05 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151880+0000", 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6350000000000006e-06 2024-09-13T10:35:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151924+0000", 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3637999999999998e-05 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.151997+0000", 2024-09-13T10:35:28.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3069000000000001e-05 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152145+0000", 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014771399999999999 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152517+0000", 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000372073 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152788+0000", 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027146800000000002 2024-09-13T10:35:28.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152819+0000", 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0953e-05 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152844+0000", 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5009000000000001e-05 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:45 2.8 2:144d5af5:::parent_id:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.152965+0000", 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.040274834000002, 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00098817999999999996, 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 45 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152965+0000", 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152965+0000", 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152966+0000", 2024-09-13T10:35:28.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2299999999999999e-07 2024-09-13T10:35:28.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152972+0000", 2024-09-13T10:35:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6899999999999997e-06 2024-09-13T10:35:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152986+0000", 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4284000000000001e-05 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.152992+0000", 2024-09-13T10:35:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0800000000000002e-06 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153032+0000", 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0392999999999998e-05 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153084+0000", 2024-09-13T10:35:28.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1282000000000001e-05 2024-09-13T10:35:28.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153315+0000", 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000231034 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153591+0000", 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027587100000000002 2024-09-13T10:35:28.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153888+0000", 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029774599999999999 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153929+0000", 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0244999999999998e-05 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.153953+0000", 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4632000000000001e-05 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:46 2.8 2:120d515b:::image5:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.154057+0000", 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.039182998000001, 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00084296700000000002, 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.372 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 46 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154057+0000", 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154057+0000", 2024-09-13T10:35:28.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154058+0000", 2024-09-13T10:35:28.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.003e-06 2024-09-13T10:35:28.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154064+0000", 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8719999999999999e-06 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154078+0000", 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4634000000000001e-05 2024-09-13T10:35:28.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154087+0000", 2024-09-13T10:35:28.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2600000000000005e-06 2024-09-13T10:35:28.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154133+0000", 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6601999999999999e-05 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154187+0000", 2024-09-13T10:35:28.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3411000000000001e-05 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154300+0000", 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113436 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154631+0000", 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033087999999999998 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154841+0000", 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021049 2024-09-13T10:35:28.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154874+0000", 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2835e-05 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.154900+0000", 2024-09-13T10:35:28.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5544000000000001e-05 2024-09-13T10:35:28.383 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.383 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.383 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:48 2.8 2:120d515b:::image5:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.155296+0000", 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.037943489, 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0082571980000000003, 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.384 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.385 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.385 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.385 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.385 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 48 2024-09-13T10:35:28.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.385 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155296+0000", 2024-09-13T10:35:28.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155296+0000", 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155297+0000", 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6799999999999999e-07 2024-09-13T10:35:28.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155300+0000", 2024-09-13T10:35:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1389999999999999e-06 2024-09-13T10:35:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155308+0000", 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3520000000000001e-06 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155312+0000", 2024-09-13T10:35:28.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2729999999999996e-06 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155348+0000", 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5929000000000001e-05 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155393+0000", 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4821999999999999e-05 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155503+0000", 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010987799999999999 2024-09-13T10:35:28.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.155765+0000", 2024-09-13T10:35:28.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026204599999999999 2024-09-13T10:35:28.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163497+0000", 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0077322179999999999 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163530+0000", 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2833000000000002e-05 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163554+0000", 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3839999999999999e-05 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.394 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.395 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.395 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:50 2.8 2:120d515b:::image5:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.395 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.163995+0000", 2024-09-13T10:35:28.395 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.029245256000003, 2024-09-13T10:35:28.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00081057099999999995, 2024-09-13T10:35:28.396 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.396 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.396 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:28.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 50 2024-09-13T10:35:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163995+0000", 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163995+0000", 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163995+0000", 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1999999999999999e-07 2024-09-13T10:35:28.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.163999+0000", 2024-09-13T10:35:28.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8419999999999998e-06 2024-09-13T10:35:28.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164007+0000", 2024-09-13T10:35:28.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.259e-06 2024-09-13T10:35:28.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164012+0000", 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7740000000000004e-06 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164052+0000", 2024-09-13T10:35:28.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9991999999999999e-05 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164107+0000", 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4769999999999999e-05 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164805+0000", 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00069821399999999995 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.404 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:51 2.8 2:120d515b:::image5:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.405 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.164937+0000", 2024-09-13T10:35:28.405 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.028302396999997, 2024-09-13T10:35:28.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00081107, 2024-09-13T10:35:28.405 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.405 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.408 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.408 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 51 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164937+0000", 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164937+0000", 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164938+0000", 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000002e-07 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164944+0000", 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6919999999999999e-06 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164958+0000", 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3791999999999999e-05 2024-09-13T10:35:28.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.164974+0000", 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5588000000000001e-05 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165018+0000", 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4790000000000003e-05 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165071+0000", 2024-09-13T10:35:28.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2200000000000002e-05 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165187+0000", 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011674400000000001 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165448+0000", 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026056499999999998 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165694+0000", 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024640799999999999 2024-09-13T10:35:28.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165725+0000", 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0457999999999999e-05 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.165749+0000", 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3866000000000002e-05 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:53 2.8 2:120d515b:::image5:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.418 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.166121+0000", 2024-09-13T10:35:28.419 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.027118510999998, 2024-09-13T10:35:28.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000786446, 2024-09-13T10:35:28.419 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.419 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166121+0000", 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166121+0000", 2024-09-13T10:35:28.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166122+0000", 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2900000000000003e-07 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166125+0000", 2024-09-13T10:35:28.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2930000000000001e-06 2024-09-13T10:35:28.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166133+0000", 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9079999999999995e-06 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166138+0000", 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3719999999999998e-06 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166174+0000", 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6193999999999999e-05 2024-09-13T10:35:28.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166223+0000", 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9215000000000002e-05 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166329+0000", 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010570599999999999 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166610+0000", 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000281296 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166856+0000", 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024623100000000002 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166885+0000", 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8243e-05 2024-09-13T10:35:28.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.166908+0000", 2024-09-13T10:35:28.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3258999999999999e-05 2024-09-13T10:35:28.429 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.429 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.430 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.430 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:55 2.8 2:120d515b:::image5:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.167270+0000", 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 54.025969865999997, 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00068995799999999996, 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.431 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 55 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167270+0000", 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167270+0000", 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167271+0000", 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2200000000000003e-07 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167274+0000", 2024-09-13T10:35:28.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.489e-06 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167282+0000", 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.011e-06 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167287+0000", 2024-09-13T10:35:28.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5109999999999997e-06 2024-09-13T10:35:28.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167319+0000", 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1828000000000001e-05 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167367+0000", 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8495999999999998e-05 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.167960+0000", 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00059280100000000001 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:122 2.8 2:13b91559:::image11:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.438 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.219983+0000", 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 53.973256755999998, 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001152101, 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.439 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 122 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.219983+0000", 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.219983+0000", 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.219984+0000", 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2800000000000005e-07 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.219988+0000", 2024-09-13T10:35:28.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2899999999999996e-06 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.220000+0000", 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2143e-05 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.220025+0000", 2024-09-13T10:35:28.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4896e-05 2024-09-13T10:35:28.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.220057+0000", 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1931999999999999e-05 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.220126+0000", 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8328e-05 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.220267+0000", 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000141328 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.220570+0000", 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00030291900000000002 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.221076+0000", 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00050655100000000005 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.221109+0000", 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449000000000001e-05 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.221135+0000", 2024-09-13T10:35:28.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6336999999999999e-05 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:130 2.8 2:16ed74a2:::image15:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.449 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.236099+0000", 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 53.957140426999999, 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001089298, 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.451 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 130 2024-09-13T10:35:28.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.451 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236099+0000", 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236099+0000", 2024-09-13T10:35:28.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236101+0000", 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9259999999999999e-06 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236111+0000", 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2900000000000008e-06 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236141+0000", 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.995e-05 2024-09-13T10:35:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236155+0000", 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3899e-05 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236228+0000", 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3021000000000004e-05 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236294+0000", 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6060000000000001e-05 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236367+0000", 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3308999999999999e-05 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.236624+0000", 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025691000000000001 2024-09-13T10:35:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237131+0000", 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00050696499999999995 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237161+0000", 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0633999999999998e-05 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237189+0000", 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7334e-05 2024-09-13T10:35:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:132 2.8 2:16ed74a2:::image15:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.237555+0000", 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 53.955684845999997, 2024-09-13T10:35:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000711996, 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 132 2024-09-13T10:35:28.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.462 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237555+0000", 2024-09-13T10:35:28.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237555+0000", 2024-09-13T10:35:28.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237556+0000", 2024-09-13T10:35:28.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5599999999999995e-07 2024-09-13T10:35:28.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237558+0000", 2024-09-13T10:35:28.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3700000000000002e-06 2024-09-13T10:35:28.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237565+0000", 2024-09-13T10:35:28.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2230000000000001e-06 2024-09-13T10:35:28.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237569+0000", 2024-09-13T10:35:28.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9230000000000004e-06 2024-09-13T10:35:28.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237603+0000", 2024-09-13T10:35:28.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3905000000000001e-05 2024-09-13T10:35:28.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237647+0000", 2024-09-13T10:35:28.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4341000000000001e-05 2024-09-13T10:35:28.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237706+0000", 2024-09-13T10:35:28.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8396000000000001e-05 2024-09-13T10:35:28.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.237963+0000", 2024-09-13T10:35:28.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025751000000000002 2024-09-13T10:35:28.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238220+0000", 2024-09-13T10:35:28.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025647099999999998 2024-09-13T10:35:28.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238246+0000", 2024-09-13T10:35:28.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6149999999999999e-05 2024-09-13T10:35:28.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238267+0000", 2024-09-13T10:35:28.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1151000000000001e-05 2024-09-13T10:35:28.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.478 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.479 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:133 2.8 2:16ed74a2:::image15:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:28.479 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:34.238368+0000", 2024-09-13T10:35:28.480 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 53.954872346999998, 2024-09-13T10:35:28.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00077251800000000003, 2024-09-13T10:35:28.480 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.480 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.481 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:28.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.482 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 133 2024-09-13T10:35:28.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.482 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238368+0000", 2024-09-13T10:35:28.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238368+0000", 2024-09-13T10:35:28.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238368+0000", 2024-09-13T10:35:28.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3e-07 2024-09-13T10:35:28.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238372+0000", 2024-09-13T10:35:28.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4060000000000001e-06 2024-09-13T10:35:28.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238380+0000", 2024-09-13T10:35:28.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3340000000000002e-06 2024-09-13T10:35:28.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238384+0000", 2024-09-13T10:35:28.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2540000000000003e-06 2024-09-13T10:35:28.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238417+0000", 2024-09-13T10:35:28.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2582000000000001e-05 2024-09-13T10:35:28.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-13T10:35:28.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238460+0000", 2024-09-13T10:35:28.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3596000000000001e-05 2024-09-13T10:35:28.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:28.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238582+0000", 2024-09-13T10:35:28.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000121449 2024-09-13T10:35:28.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-13T10:35:28.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.238859+0000", 2024-09-13T10:35:28.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000276765 2024-09-13T10:35:28.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:28.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239089+0000", 2024-09-13T10:35:28.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000230473 2024-09-13T10:35:28.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-13T10:35:28.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239118+0000", 2024-09-13T10:35:28.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9292e-05 2024-09-13T10:35:28.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:34.239140+0000", 2024-09-13T10:35:28.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1637e-05 2024-09-13T10:35:28.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.498 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.24508.0:185 2.8 2:157ad5b2:::doesnotexist:head [call rbd.get_parent in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:28.499 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-13T10:34:35.275000+0000", 2024-09-13T10:35:28.499 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 52.918240208, 2024-09-13T10:35:28.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001427111, 2024-09-13T10:35:28.500 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-13T10:35:28.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-13T10:35:28.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-13T10:35:28.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-13T10:35:28.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.24508", 2024-09-13T10:35:28.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:28.501 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 185 2024-09-13T10:35:28.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.502 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-13T10:35:28.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-13T10:35:28.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275000+0000", 2024-09-13T10:35:28.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-13T10:35:28.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275000+0000", 2024-09-13T10:35:28.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-13T10:35:28.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-13T10:35:28.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275001+0000", 2024-09-13T10:35:28.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.778e-06 2024-09-13T10:35:28.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-13T10:35:28.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275009+0000", 2024-09-13T10:35:28.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4889999999999998e-06 2024-09-13T10:35:28.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-13T10:35:28.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275027+0000", 2024-09-13T10:35:28.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7935e-05 2024-09-13T10:35:28.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-13T10:35:28.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275039+0000", 2024-09-13T10:35:28.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2397e-05 2024-09-13T10:35:28.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275109+0000", 2024-09-13T10:35:28.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9309999999999999e-05 2024-09-13T10:35:28.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-13T10:35:28.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.275113+0000", 2024-09-13T10:35:28.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4290000000000003e-06 2024-09-13T10:35:28.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276358+0000", 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0012448139999999999 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-13T10:35:28.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-13T10:34:35.276427+0000", 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8959000000000002e-05 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-13T10:35:28.513 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-13T10:35:28.514 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:28 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4150676051' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-13T10:35:28.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:28 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4150676051' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-13T10:35:28.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:28 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4150676051' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-13T10:35:28.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 2 1 2024-09-13T10:35:29.093 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.8 dump_ops_in_flight 2024-09-13T10:35:29.168 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:29.349 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:29.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:29 smithi138 ceph-mon[31033]: pgmap v366: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:29.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:29 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4150676051' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-13T10:35:29.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:29 smithi138 ceph-mon[31033]: osdmap e129: 12 total, 12 up, 12 in 2024-09-13T10:35:29.470 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:29 smithi173 ceph-mon[30864]: pgmap v366: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:29.471 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:29 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4150676051' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-13T10:35:29.471 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:29 smithi173 ceph-mon[30864]: osdmap e129: 12 total, 12 up, 12 in 2024-09-13T10:35:29.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:29 smithi060 ceph-mon[27375]: pgmap v366: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:29.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:29 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4150676051' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-13T10:35:29.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:29 smithi060 ceph-mon[27375]: osdmap e129: 12 total, 12 up, 12 in 2024-09-13T10:35:29.879 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:29.879 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-13T10:35:29.879 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:30.225 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T10:35:30.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:30 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:30.508 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:30 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:30.508 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:30 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4098664075' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-13T10:35:30.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.8 dump_blocked_ops 2024-09-13T10:35:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:30 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:30 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:30 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4098664075' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-13T10:35:30.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:30 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:30.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:30 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:30.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:30 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4098664075' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-13T10:35:30.778 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:30.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-13T10:35:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:31.326 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-13T10:35:31.326 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-13T10:35:31.326 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:31 smithi173 ceph-mon[30864]: pgmap v368: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:31 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4098664075' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-13T10:35:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:31 smithi173 ceph-mon[30864]: osdmap e130: 12 total, 12 up, 12 in 2024-09-13T10:35:31.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:31 smithi060 ceph-mon[27375]: pgmap v368: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:31.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:31 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4098664075' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-13T10:35:31.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:31 smithi060 ceph-mon[27375]: osdmap e130: 12 total, 12 up, 12 in 2024-09-13T10:35:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:31 smithi138 ceph-mon[31033]: pgmap v368: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:31 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4098664075' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-13T10:35:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:31 smithi138 ceph-mon[31033]: osdmap e130: 12 total, 12 up, 12 in 2024-09-13T10:35:31.975 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.8 dump_historic_ops 2024-09-13T10:35:32.213 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:32.231 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T10:35:32.521 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:32 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2496936789' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-13T10:35:32.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:32 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2496936789' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-13T10:35:32.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:32 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2496936789' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-13T10:35:32.760 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:32.760 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-13T10:35:32.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-13T10:35:32.760 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-13T10:35:32.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.760 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:9 2.f 2:f586ebe2:::image2:head [call rbd.copyup in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.625581+0000", 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 59.131024955999997, 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.014172123999999999, 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:32.761 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 9 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.625581+0000", 2024-09-13T10:35:32.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.625581+0000", 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.625582+0000", 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9400000000000004e-07 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.627798+0000", 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0022166629999999998 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.627823+0000", 2024-09-13T10:35:32.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4975999999999999e-05 2024-09-13T10:35:32.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.627833+0000", 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1439999999999992e-06 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.627894+0000", 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0903000000000003e-05 2024-09-13T10:35:32.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.627956+0000", 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2279000000000005e-05 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.628237+0000", 2024-09-13T10:35:32.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028144500000000001 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.634211+0000", 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0059740410000000002 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639666+0000", 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0054543830000000001 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639702+0000", 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6400999999999999e-05 2024-09-13T10:35:32.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639753+0000", 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0995000000000002e-05 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:10 2.f 2:f586ebe2:::image2:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.639879+0000", 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 59.116727197000003, 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0029651080000000002, 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:32.771 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 10 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639879+0000", 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639879+0000", 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639880+0000", 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9000000000000005e-07 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639884+0000", 2024-09-13T10:35:32.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7249999999999997e-06 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639894+0000", 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9310000000000003e-06 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639901+0000", 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0060000000000003e-06 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639952+0000", 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0507999999999998e-05 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.640041+0000", 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8658000000000004e-05 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.642844+0000", 2024-09-13T10:35:32.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0028032899999999999 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:11 2.f 2:f586ebe2:::image2:head [call rbd.copyup in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.710223+0000", 2024-09-13T10:35:32.778 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 59.046382512999998, 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0020548150000000002, 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 11 2024-09-13T10:35:32.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.710223+0000", 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.710223+0000", 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.710225+0000", 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.894e-06 2024-09-13T10:35:32.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711222+0000", 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00099686200000000005 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711245+0000", 2024-09-13T10:35:32.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3348000000000001e-05 2024-09-13T10:35:32.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.783 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.783 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711272+0000", 2024-09-13T10:35:32.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6546999999999998e-05 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711361+0000", 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8561999999999997e-05 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711476+0000", 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000115318 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.712278+0000", 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000802284 2024-09-13T10:35:32.785 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:13 2.f 2:f586ebe2:::image2:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.714912+0000", 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 59.041694008, 2024-09-13T10:35:32.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001110988, 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 13 2024-09-13T10:35:32.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.714912+0000", 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.714912+0000", 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.714914+0000", 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.745e-06 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.714923+0000", 2024-09-13T10:35:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.6730000000000004e-06 2024-09-13T10:35:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.790 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.714944+0000", 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1013999999999999e-05 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.714971+0000", 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6832e-05 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715033+0000", 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1672000000000002e-05 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715095+0000", 2024-09-13T10:35:32.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2304999999999994e-05 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 9", 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715187+0000", 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1564999999999995e-05 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715701+0000", 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00051459100000000001 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715962+0000", 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000260556 2024-09-13T10:35:32.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715992+0000", 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0062000000000001e-05 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.716023+0000", 2024-09-13T10:35:32.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0973000000000003e-05 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:771 2.6 e102/97 2:67cae2ed:::rbd_mirroring:head v 102'295, mlcod=102'293)", 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:41.377681+0000", 2024-09-13T10:35:32.796 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 51.378925207000002, 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.16278577599999999, 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.377681+0000", 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.377681+0000", 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.377681+0000", 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2900000000000004e-07 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.377702+0000", 2024-09-13T10:35:32.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0908999999999999e-05 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.539814+0000", 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.162111437 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.539823+0000", 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7510000000000003e-06 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.539957+0000", 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013409699999999999 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.539979+0000", 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1719e-05 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.540420+0000", 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00044118100000000002 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:41.540466+0000", 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6152999999999997e-05 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.803 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:889 2.6 e104/97 2:67cae2ed:::rbd_mirroring:head v 104'413, mlcod=104'411)", 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:42.693311+0000", 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 50.063294827, 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28828921800000001, 2024-09-13T10:35:32.804 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.693311+0000", 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.805 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.693311+0000", 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.693312+0000", 2024-09-13T10:35:32.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.693316+0000", 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5990000000000004e-06 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.980762+0000", 2024-09-13T10:35:32.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.28744576599999999 2024-09-13T10:35:32.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.980768+0000", 2024-09-13T10:35:32.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7520000000000002e-06 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.980856+0000", 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.8443999999999995e-05 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.980880+0000", 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4199e-05 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.981509+0000", 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00062861999999999998 2024-09-13T10:35:32.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.981600+0000", 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1155000000000004e-05 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:1299 2.6 e106/97 2:67cae2ed:::rbd_mirroring:head v 106'823, mlcod=106'821)", 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:45.696381+0000", 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 47.060225275999997, 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042678094, 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.813 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.696381+0000", 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.696381+0000", 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.696382+0000", 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2899999999999999e-06 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.696391+0000", 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3500000000000003e-06 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.738594+0000", 2024-09-13T10:35:32.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.042202298999999999 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.738598+0000", 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5179999999999998e-06 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.738674+0000", 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.6260000000000005e-05 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.738686+0000", 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2152e-05 2024-09-13T10:35:32.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.739029+0000", 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034219600000000001 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.739059+0000", 2024-09-13T10:35:32.820 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0029e-05 2024-09-13T10:35:32.820 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.820 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.820 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.820 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:1439 2.6 e106/97 2:67cae2ed:::rbd_mirroring:head v 106'963, mlcod=106'961)", 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:45.898309+0000", 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 46.858296778000003, 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.728553267, 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.821 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.898309+0000", 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.822 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.898309+0000", 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.898310+0000", 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4499999999999997e-07 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.823 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.824 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:45.898313+0000", 2024-09-13T10:35:32.824 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.067e-06 2024-09-13T10:35:32.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.824 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.824 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.625897+0000", 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.72758422899999997 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.625911+0000", 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.376e-05 2024-09-13T10:35:32.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.626056+0000", 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014543999999999999 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.826 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.626097+0000", 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1128999999999999e-05 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.626832+0000", 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00073442500000000003 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.827 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.626862+0000", 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0772000000000003e-05 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.828 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:1440 2.6 e106/97 2:67cae2ed:::rbd_mirroring:head v 106'964, mlcod=106'962)", 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:46.891980+0000", 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 45.864625611000001, 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001103627, 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.829 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.891980+0000", 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.891980+0000", 2024-09-13T10:35:32.830 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.891983+0000", 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5299999999999999e-06 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.831 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.892002+0000", 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9049999999999999e-05 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.892086+0000", 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3834999999999999e-05 2024-09-13T10:35:32.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.892103+0000", 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7181999999999999e-05 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.833 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.892248+0000", 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014521400000000001 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.892292+0000", 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3931999999999999e-05 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.834 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.893035+0000", 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00074291699999999995 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:46.893084+0000", 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8967000000000001e-05 2024-09-13T10:35:32.835 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:1830 2.6 e108/97 2:67cae2ed:::rbd_mirroring:head v 108'1354, mlcod=108'1352)", 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:53.815812+0000", 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 38.940794265000001, 2024-09-13T10:35:32.836 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.34385415899999999, 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:53.815812+0000", 2024-09-13T10:35:32.837 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.838 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.838 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.838 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:53.815812+0000", 2024-09-13T10:35:32.838 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:53.815812+0000", 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1099999999999995e-07 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.839 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:53.815818+0000", 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.4079999999999997e-06 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.158520+0000", 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.34270227800000003 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.840 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.158535+0000", 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4593000000000001e-05 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.158679+0000", 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014393699999999999 2024-09-13T10:35:32.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.158718+0000", 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9595999999999998e-05 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.842 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.159622+0000", 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00090409099999999999 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.159666+0000", 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3544999999999997e-05 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.843 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:1950 2.6 e108/97 2:67cae2ed:::rbd_mirroring:head v 108'1474, mlcod=108'1472)", 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:54.340373+0000", 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 38.416232675000003, 2024-09-13T10:35:32.844 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010930460000000001, 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.845 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340373+0000", 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340373+0000", 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.846 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340375+0000", 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.373e-06 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340384+0000", 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.8230000000000006e-06 2024-09-13T10:35:32.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340455+0000", 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.0525999999999995e-05 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.848 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340471+0000", 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5716e-05 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340596+0000", 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012527099999999999 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.849 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.340653+0000", 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7265000000000001e-05 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.341436+0000", 2024-09-13T10:35:32.850 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00078268100000000002 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.341466+0000", 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0391000000000001e-05 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.851 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.852 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.852 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.852 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2122 2.6 e108/97 2:67cae2ed:::rbd_mirroring:head v 108'1646, mlcod=108'1644)", 2024-09-13T10:35:32.852 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:54.605115+0000", 2024-09-13T10:35:32.852 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 38.151490684999999, 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.094945753999999993, 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.853 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.605115+0000", 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.605115+0000", 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.854 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.605117+0000", 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5430000000000001e-06 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.605127+0000", 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0607e-05 2024-09-13T10:35:32.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.698970+0000", 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.093842677999999999 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.856 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.698991+0000", 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0625999999999999e-05 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.699174+0000", 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000183783 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.857 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.699215+0000", 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0611999999999997e-05 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.700029+0000", 2024-09-13T10:35:32.858 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00081396300000000001 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:54.700061+0000", 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1942e-05 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2192 2.6 e109/97 2:67cae2ed:::rbd_mirroring:head v 109'1716, mlcod=108'1714)", 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:55.749231+0000", 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 37.007374675000001, 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001350303, 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.860 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749231+0000", 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.861 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749231+0000", 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749232+0000", 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.37e-07 2024-09-13T10:35:32.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749239+0000", 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.25e-06 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.863 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749310+0000", 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1181999999999997e-05 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749330+0000", 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0149e-05 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.864 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749486+0000", 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000155754 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.749526+0000", 2024-09-13T10:35:32.865 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9481999999999999e-05 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.750541+0000", 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010159100000000001 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.866 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.750582+0000", 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0039000000000001e-05 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.867 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2203 2.6 e109/97 2:67cae2ed:::rbd_mirroring:head v 109'1727, mlcod=109'1725)", 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:55.767364+0000", 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 36.989241516, 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001130409, 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.868 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767364+0000", 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767364+0000", 2024-09-13T10:35:32.869 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767365+0000", 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.75e-07 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.870 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767370+0000", 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.853e-06 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767390+0000", 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0596000000000001e-05 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.871 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767397+0000", 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.63e-06 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767453+0000", 2024-09-13T10:35:32.872 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6431999999999998e-05 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.767468+0000", 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4606e-05 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.873 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.768463+0000", 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00099540099999999992 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.768495+0000", 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1316000000000003e-05 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.874 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2281 2.6 e109/97 2:67cae2ed:::rbd_mirroring:head v 109'1805, mlcod=109'1803)", 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:55.876391+0000", 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 36.880215034000003, 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001092995, 2024-09-13T10:35:32.875 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876391+0000", 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.876 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876391+0000", 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.877 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876392+0000", 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3769999999999999e-06 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876401+0000", 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7199999999999995e-06 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.878 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876472+0000", 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1027999999999998e-05 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876484+0000", 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.188e-05 2024-09-13T10:35:32.879 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876571+0000", 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.7501000000000001e-05 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.880 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.876605+0000", 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3556999999999997e-05 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.877451+0000", 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00084631099999999998 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.881 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:55.877484+0000", 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2620999999999998e-05 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.882 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2339 2.6 e110/97 2:67cae2ed:::rbd_mirroring:head v 110'1863, mlcod=109'1861)", 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:56.833250+0000", 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 35.923355411999999, 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001178405, 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.883 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833250+0000", 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.884 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833250+0000", 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833253+0000", 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1969999999999999e-06 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.885 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833269+0000", 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6589999999999999e-05 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833355+0000", 2024-09-13T10:35:32.886 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5834999999999994e-05 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833371+0000", 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6311999999999999e-05 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.887 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833510+0000", 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000139054 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.833558+0000", 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7704000000000003e-05 2024-09-13T10:35:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.834393+0000", 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00083476000000000004 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.889 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:56.834429+0000", 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5952999999999999e-05 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2475 2.6 e110/97 2:67cae2ed:::rbd_mirroring:head v 110'1999, mlcod=110'1997)", 2024-09-13T10:35:32.890 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:57.839212+0000", 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.917393392000001, 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010902609999999999, 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.891 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839212+0000", 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839212+0000", 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839215+0000", 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1399999999999998e-06 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839241+0000", 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5885e-05 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839276+0000", 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5747000000000002e-05 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839301+0000", 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4797999999999999e-05 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839360+0000", 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8647000000000003e-05 2024-09-13T10:35:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.839378+0000", 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8108999999999999e-05 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.840271+0000", 2024-09-13T10:35:32.897 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00089362800000000004 2024-09-13T10:35:32.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.897 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.897 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.900 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.840303+0000", 2024-09-13T10:35:32.900 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1306999999999997e-05 2024-09-13T10:35:32.900 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2518 2.6 e110/97 2:67cae2ed:::rbd_mirroring:head v 110'2042, mlcod=110'2040)", 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:57.901249+0000", 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.855357216000002, 2024-09-13T10:35:32.901 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.95275701300000004, 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.901249+0000", 2024-09-13T10:35:32.902 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.901249+0000", 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.903 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.901250+0000", 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2100000000000001e-06 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:57.901258+0000", 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5669999999999995e-06 2024-09-13T10:35:32.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.852826+0000", 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.95156759400000002 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.852834+0000", 2024-09-13T10:35:32.905 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7600000000000002e-06 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.852914+0000", 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0451000000000006e-05 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.906 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.852937+0000", 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2599000000000002e-05 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.853957+0000", 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001020284 2024-09-13T10:35:32.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.854006+0000", 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8547999999999997e-05 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.908 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2530 2.6 e111/97 2:67cae2ed:::rbd_mirroring:head v 111'2054, mlcod=111'2052)", 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:58.879821+0000", 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.876784749999999, 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001122519, 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.909 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879821+0000", 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.910 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879821+0000", 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879822+0000", 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6300000000000004e-07 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.911 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879829+0000", 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6390000000000003e-06 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879859+0000", 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0352999999999999e-05 2024-09-13T10:35:32.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879865+0000", 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3160000000000002e-06 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.913 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879936+0000", 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1058999999999998e-05 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.879961+0000", 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4318999999999999e-05 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.914 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.880887+0000", 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00092585300000000003 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:58.880944+0000", 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7117000000000002e-05 2024-09-13T10:35:32.915 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:2575 2.6 e111/97 2:67cae2ed:::rbd_mirroring:head v 111'2095, mlcod=111'2093)", 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:59.029417+0000", 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 33.727188542999997, 2024-09-13T10:35:32.916 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0037544330000000002, 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029417+0000", 2024-09-13T10:35:32.917 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029417+0000", 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029419+0000", 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7829999999999999e-06 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029451+0000", 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1582999999999999e-05 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.919 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029517+0000", 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6218999999999997e-05 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:32.920 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029535+0000", 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7802999999999999e-05 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029707+0000", 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017271699999999999 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.921 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.029723+0000", 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5498e-05 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.033138+0000", 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00341472 2024-09-13T10:35:32.922 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:59.033172+0000", 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4109999999999997e-05 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.923 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:32.924 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:32.924 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:32.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 3 1 2024-09-13T10:35:33.184 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:33.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:33 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:33.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:33 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:33.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:33 smithi173 ceph-mon[30864]: pgmap v370: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:33.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:33 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2496936789' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-13T10:35:33.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:33 smithi173 ceph-mon[30864]: osdmap e131: 12 total, 12 up, 12 in 2024-09-13T10:35:33.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.9 dump_ops_in_flight 2024-09-13T10:35:33.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:33 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:33.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:33 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:33.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:33 smithi060 ceph-mon[27375]: pgmap v370: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:33.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:33 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2496936789' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-13T10:35:33.582 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:33 smithi060 ceph-mon[27375]: osdmap e131: 12 total, 12 up, 12 in 2024-09-13T10:35:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:33 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:33 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:33 smithi138 ceph-mon[31033]: pgmap v370: 21 pgs: 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:33 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2496936789' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-13T10:35:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:33 smithi138 ceph-mon[31033]: osdmap e131: 12 total, 12 up, 12 in 2024-09-13T10:35:33.776 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:34.238 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T10:35:34.309 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:34.309 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:34.309 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-13T10:35:34.309 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:34.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:34 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2305996860' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-13T10:35:34.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:34 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2305996860' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-13T10:35:34.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:34 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2305996860' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-13T10:35:34.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-13T10:35:34.942 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.9 dump_blocked_ops 2024-09-13T10:35:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:35.189 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: pgmap v372: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2305996860' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: osdmap e132: 12 total, 12 up, 12 in 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: osdmap e133: 12 total, 12 up, 12 in 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:35.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:35.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:35:35.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:35.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:35.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:35.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:35 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:35:35.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: pgmap v372: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:35.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2305996860' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-13T10:35:35.554 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: osdmap e132: 12 total, 12 up, 12 in 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: osdmap e133: 12 total, 12 up, 12 in 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:35.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:35:35.556 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:35.556 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:35.556 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:35.556 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:35 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:35:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: pgmap v372: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2305996860' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: osdmap e132: 12 total, 12 up, 12 in 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: osdmap e133: 12 total, 12 up, 12 in 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:35.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:35:35.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:35:35.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:35:35.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:35:35.602 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:35 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:35:35.683 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:35.683 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:35.683 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-13T10:35:35.683 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-13T10:35:35.683 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:36.246 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T10:35:36.462 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:36 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3004004263' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-13T10:35:36.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.9 dump_historic_ops 2024-09-13T10:35:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:36 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3004004263' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-13T10:35:36.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:36 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3004004263' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-13T10:35:36.740 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:36.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 4 1 2024-09-13T10:35:37.192 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:37.232 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.f epoch 98/97 rep_tid 0 entries 98'1 (0'0) error 2:f586ebe2:::image2:head by client.24508.0:4 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.555348+0000", 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.674331527, 2024-09-13T10:35:37.233 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00036926999999999999, 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.234 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555348+0000", 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555348+0000", 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.235 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555348+0000", 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1999999999999999e-07 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555363+0000", 2024-09-13T10:35:37.236 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.489e-05 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555403+0000", 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9748e-05 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.237 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555413+0000", 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0257e-05 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555485+0000", 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1470000000000005e-05 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.238 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.555717+0000", 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000232785 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.239 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:5 2.f e98/97 2:f586ebe2:::image2:head v 98'2, mlcod=0'0)", 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.556460+0000", 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.673219406000001, 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00039544200000000003, 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.240 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556460+0000", 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556460+0000", 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.241 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556461+0000", 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.06e-07 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556467+0000", 2024-09-13T10:35:37.242 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1500000000000004e-06 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556494+0000", 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7157000000000001e-05 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.243 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556499+0000", 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2970000000000003e-06 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556537+0000", 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7833999999999997e-05 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.244 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556550+0000", 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2424e-05 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.245 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556812+0000", 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026262799999999999 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.556856+0000", 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3346000000000001e-05 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.246 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:7 2.f e98/97 2:f586ebe2:::image2:head v 98'3, mlcod=98'1)", 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.623613+0000", 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.606066446, 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045720899999999998, 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.247 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623613+0000", 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.248 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623613+0000", 2024-09-13T10:35:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.249 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623614+0000", 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.47e-07 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623629+0000", 2024-09-13T10:35:37.250 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4842e-05 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623663+0000", 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3725e-05 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.251 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623670+0000", 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1520000000000003e-06 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623741+0000", 2024-09-13T10:35:37.252 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1686000000000004e-05 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.623764+0000", 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2126000000000001e-05 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.253 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.624029+0000", 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026563900000000002 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.624070+0000", 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1192000000000001e-05 2024-09-13T10:35:37.254 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:9 2.f e98/97 2:f586ebe2:::image2:head v 98'5, mlcod=98'3)", 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.628358+0000", 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.601321976000001, 2024-09-13T10:35:37.255 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.011137662, 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.628358+0000", 2024-09-13T10:35:37.256 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.628358+0000", 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.257 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.258 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.258 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.628360+0000", 2024-09-13T10:35:37.258 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1569999999999998e-06 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.631635+0000", 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0032754210000000001 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.259 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.631716+0000", 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0840000000000005e-05 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.631731+0000", 2024-09-13T10:35:37.260 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4987999999999999e-05 2024-09-13T10:35:37.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.261 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.261 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.261 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.631809+0000", 2024-09-13T10:35:37.261 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7769999999999996e-05 2024-09-13T10:35:37.261 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.631829+0000", 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0576000000000001e-05 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.262 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639454+0000", 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0076247529999999997 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.639495+0000", 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.1156999999999999e-05 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.263 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.f epoch 98/97 rep_tid 5 entries 98'6 (0'0) error 2:f586ebe2:::image2:head by client.24508.0:11 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 98'4)", 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.711693+0000", 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.517986379999996, 2024-09-13T10:35:37.264 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000388224, 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.265 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711693+0000", 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711693+0000", 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.266 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711694+0000", 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.192e-06 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711706+0000", 2024-09-13T10:35:37.267 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1457999999999999e-05 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711745+0000", 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9310999999999997e-05 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.268 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711755+0000", 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9159999999999996e-06 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.711844+0000", 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9152000000000003e-05 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.269 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.712081+0000", 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00023719499999999999 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.270 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:13 2.f e98/97 2:f586ebe2:::image2:head v 98'7, mlcod=98'5)", 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:33.715304+0000", 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.514375301000001, 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00055334500000000001, 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.271 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715304+0000", 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.272 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715304+0000", 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715305+0000", 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1149999999999999e-06 2024-09-13T10:35:37.273 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715314+0000", 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6119999999999999e-06 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.274 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715354+0000", 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0361000000000002e-05 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715363+0000", 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2339999999999994e-06 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.275 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715437+0000", 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.4432999999999997e-05 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715462+0000", 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.508e-05 2024-09-13T10:35:37.276 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715824+0000", 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00036213899999999998 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.277 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:33.715858+0000", 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3371000000000003e-05 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:41 2.8 e98/97 2:120d515b:::image5:head v 98'1, mlcod=0'0)", 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.149611+0000", 2024-09-13T10:35:37.278 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.080069016000003, 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048880500000000003, 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.279 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149611+0000", 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149611+0000", 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.280 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149612+0000", 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.7100000000000011e-07 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149620+0000", 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9820000000000008e-06 2024-09-13T10:35:37.281 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.282 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.282 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.282 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149654+0000", 2024-09-13T10:35:37.282 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.468e-05 2024-09-13T10:35:37.282 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.283 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.283 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149665+0000", 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0299000000000001e-05 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149728+0000", 2024-09-13T10:35:37.284 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2985999999999996e-05 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.149744+0000", 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6629999999999998e-05 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.285 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.150060+0000", 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031580399999999999 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.286 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.150099+0000", 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9453000000000003e-05 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.287 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.8 epoch 98/97 rep_tid 1 entries 98'2 (0'0) error 2:120d515b:::image5:head by client.24508.0:43 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.151152+0000", 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.078527893999997, 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00040606099999999999, 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:37.288 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151152+0000", 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.289 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151152+0000", 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151152+0000", 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:35:37.290 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151161+0000", 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.4409999999999993e-06 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.291 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151199+0000", 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8349999999999997e-05 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151214+0000", 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4579000000000001e-05 2024-09-13T10:35:37.292 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151291+0000", 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.7515000000000006e-05 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.293 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.151558+0000", 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026649900000000001 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:44 2.8 e98/97 2:144d5af5:::parent_id:head v 98'3, mlcod=98'1)", 2024-09-13T10:35:37.294 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.152248+0000", 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.077431322000002, 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00043762100000000002, 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.295 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152248+0000", 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152248+0000", 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.296 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152249+0000", 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9299999999999998e-07 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.297 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152254+0000", 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2630000000000003e-06 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152280+0000", 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5474e-05 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.298 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152286+0000", 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6900000000000003e-06 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.299 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152340+0000", 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3946999999999997e-05 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152362+0000", 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1671e-05 2024-09-13T10:35:37.300 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152649+0000", 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028751599999999998 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.301 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.152686+0000", 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6467000000000002e-05 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:45 2.8 e98/97 2:144d5af5:::parent_id:head v 98'4, mlcod=98'2)", 2024-09-13T10:35:37.302 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.153408+0000", 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.076272060999997, 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000379589, 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.303 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153408+0000", 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153408+0000", 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.304 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153408+0000", 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2399999999999998e-07 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.305 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153414+0000", 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0549999999999996e-06 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153441+0000", 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6659999999999999e-05 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.306 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153446+0000", 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2800000000000003e-06 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153494+0000", 2024-09-13T10:35:37.307 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7642999999999998e-05 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153515+0000", 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1443000000000001e-05 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.308 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153755+0000", 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00023962699999999999 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.309 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.153787+0000", 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2257e-05 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:48 2.8 e98/97 2:120d515b:::image5:head v 98'6, mlcod=98'4)", 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.162718+0000", 2024-09-13T10:35:37.310 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.066961474999999, 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00045613700000000001, 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.311 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162718+0000", 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162718+0000", 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.312 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162719+0000", 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.0299999999999998e-07 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.313 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162724+0000", 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0000000000000004e-06 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162744+0000", 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0381e-05 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.314 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162751+0000", 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2990000000000002e-06 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162809+0000", 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8678000000000003e-05 2024-09-13T10:35:37.315 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.316 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.316 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.316 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.162825+0000", 2024-09-13T10:35:37.316 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5213e-05 2024-09-13T10:35:37.316 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.163143+0000", 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031857700000000001 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.317 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.163174+0000", 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1186000000000003e-05 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.318 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:122 2.8 e98/97 2:13b91559:::image11:head v 98'11, mlcod=98'9)", 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.220495+0000", 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 63.009184654999999, 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000434435, 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.319 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220495+0000", 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220495+0000", 2024-09-13T10:35:37.320 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220497+0000", 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5400000000000001e-06 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.321 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220511+0000", 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4054e-05 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220568+0000", 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7222999999999997e-05 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.322 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220573+0000", 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0259999999999998e-06 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220612+0000", 2024-09-13T10:35:37.323 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9134999999999997e-05 2024-09-13T10:35:37.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.324 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.324 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.324 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220623+0000", 2024-09-13T10:35:37.324 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0540000000000001e-05 2024-09-13T10:35:37.324 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220898+0000", 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00027593300000000002 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.325 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.220929+0000", 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0984e-05 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:130 2.8 e98/97 2:16ed74a2:::image15:head v 98'12, mlcod=98'10)", 2024-09-13T10:35:37.326 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.236550+0000", 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.993129336000003, 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00048116599999999999, 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.327 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236550+0000", 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236550+0000", 2024-09-13T10:35:37.328 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236552+0000", 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4500000000000001e-06 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.329 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236562+0000", 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.023e-05 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236622+0000", 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9738999999999998e-05 2024-09-13T10:35:37.330 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236636+0000", 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4566000000000001e-05 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.331 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236732+0000", 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5687999999999996e-05 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.236762+0000", 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0128999999999999e-05 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.332 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.237007+0000", 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024440900000000001 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.237031+0000", 2024-09-13T10:35:37.333 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4955000000000001e-05 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:142 2.d 2:b1b22a0f:::image17:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.254382+0000", 2024-09-13T10:35:37.334 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.975297603999998, 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0012195820000000001, 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:37.335 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 142 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254382+0000", 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.336 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254382+0000", 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254384+0000", 2024-09-13T10:35:37.337 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9870000000000002e-06 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254395+0000", 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.08e-05 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.338 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254416+0000", 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0729000000000001e-05 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254454+0000", 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.888e-05 2024-09-13T10:35:37.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254510+0000", 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5365999999999998e-05 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.340 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.254581+0000", 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.1327000000000003e-05 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255602+0000", 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0010204929999999999 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.341 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:143 2.d 2:b1b22a0f:::image17:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.255691+0000", 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.973988824999999, 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001055956, 2024-09-13T10:35:37.342 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 143 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.343 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255691+0000", 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.344 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255691+0000", 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255691+0000", 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.99e-07 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.345 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255695+0000", 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5420000000000001e-06 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255705+0000", 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0402e-05 2024-09-13T10:35:37.346 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255712+0000", 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4400000000000002e-06 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255749+0000", 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7596999999999999e-05 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255795+0000", 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5701000000000002e-05 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 4", 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.255925+0000", 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000129846 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.256219+0000", 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029385099999999998 2024-09-13T10:35:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.256695+0000", 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00047629399999999999 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.256723+0000", 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8217000000000001e-05 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.256747+0000", 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3467000000000001e-05 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:145 2.d 2:b1b22a0f:::image17:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.257077+0000", 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.972602664, 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.001277919, 2024-09-13T10:35:37.352 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 145 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.353 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257077+0000", 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.354 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257077+0000", 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257078+0000", 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6599999999999996e-07 2024-09-13T10:35:37.355 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257080+0000", 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7999999999999999e-06 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.356 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257088+0000", 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2620000000000001e-06 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257100+0000", 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1982e-05 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.357 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257133+0000", 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3594999999999999e-05 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257178+0000", 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4960000000000003e-05 2024-09-13T10:35:37.358 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 4", 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257278+0000", 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.9572000000000001e-05 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:37.359 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.257532+0000", 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025455900000000001 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258306+0000", 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00077343400000000004 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.360 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258332+0000", 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6599000000000001e-05 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258355+0000", 2024-09-13T10:35:37.361 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2589999999999999e-05 2024-09-13T10:35:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.362 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:147 2.d 2:b1b22a0f:::image17:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.258690+0000", 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 62.970990082, 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00087630800000000001, 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:37.363 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 147 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258690+0000", 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.364 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258690+0000", 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.365 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258690+0000", 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.51e-07 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258694+0000", 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.698e-06 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.366 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258701+0000", 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.43e-06 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258713+0000", 2024-09-13T10:35:37.367 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1426999999999999e-05 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258747+0000", 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3828000000000002e-05 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.368 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258791+0000", 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4227999999999997e-05 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 4", 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.258885+0000", 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.4446999999999999e-05 2024-09-13T10:35:37.369 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.259107+0000", 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00022143799999999999 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:37.370 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.259519+0000", 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000412249 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.259545+0000", 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5789999999999999e-05 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.371 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.259566+0000", 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1222000000000001e-05 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.372 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:186 2.14 e99/97 2:2ea5a297:::image21:head v 99'1, mlcod=0'0)", 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.277186+0000", 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 61.952493681999997, 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00039631099999999999, 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:37.373 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277186+0000", 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.374 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277186+0000", 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277186+0000", 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7100000000000001e-07 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.375 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277191+0000", 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.1510000000000003e-06 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277227+0000", 2024-09-13T10:35:37.376 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5583999999999997e-05 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277234+0000", 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2259999999999999e-06 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.377 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277285+0000", 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0739000000000003e-05 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277298+0000", 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3009e-05 2024-09-13T10:35:37.378 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277550+0000", 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025249799999999998 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.277582+0000", 2024-09-13T10:35:37.379 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1933000000000001e-05 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.14 epoch 99/97 rep_tid 34 entries 99'2 (0'0) error 2:2ea5a297:::image21:head by client.24508.0:188 0.000000 -8 [r=-8+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.278934+0000", 2024-09-13T10:35:37.380 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 61.950745478999998, 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035858299999999999, 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.381 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.278934+0000", 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.278934+0000", 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.382 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.278935+0000", 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6599999999999996e-07 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.278944+0000", 2024-09-13T10:35:37.383 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9360000000000005e-06 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.278971+0000", 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7764e-05 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.384 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.278980+0000", 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1829999999999994e-06 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.279069+0000", 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9165000000000004e-05 2024-09-13T10:35:37.385 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.279293+0000", 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000223969 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.386 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPull(2.8 e104/103 cost 1049576)", 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:42.825255+0000", 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 54.404424204000001, 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00046919600000000001, 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:37.387 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825255+0000", 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.388 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825255+0000", 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825257+0000", 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4470000000000001e-06 2024-09-13T10:35:37.389 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825275+0000", 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7706e-05 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:37.390 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825331+0000", 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5952000000000002e-05 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825347+0000", 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6799999999999998e-05 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.391 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825464+0000", 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00011658999999999999 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:42.825725+0000", 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026070100000000001 2024-09-13T10:35:37.392 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:37.393 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:37.393 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:37 smithi173 ceph-mon[30864]: pgmap v375: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:37 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3004004263' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-13T10:35:37.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:37 smithi173 ceph-mon[30864]: osdmap e134: 12 total, 12 up, 12 in 2024-09-13T10:35:37.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:37 smithi060 ceph-mon[27375]: pgmap v375: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:37.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:37 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3004004263' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-13T10:35:37.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:37 smithi060 ceph-mon[27375]: osdmap e134: 12 total, 12 up, 12 in 2024-09-13T10:35:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:37 smithi138 ceph-mon[31033]: pgmap v375: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:37.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:37 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3004004263' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-13T10:35:37.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:37 smithi138 ceph-mon[31033]: osdmap e134: 12 total, 12 up, 12 in 2024-09-13T10:35:38.116 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.10 dump_ops_in_flight 2024-09-13T10:35:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T10:35:38.360 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:38.481 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:38 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/685398622' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-13T10:35:38.557 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:38 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/685398622' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-13T10:35:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:38 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/685398622' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-13T10:35:38.874 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:38.875 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:38.875 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-13T10:35:38.875 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:38.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-13T10:35:39.183 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:39.524 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:39 smithi173 ceph-mon[30864]: pgmap v377: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:39.524 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:39 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/685398622' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-13T10:35:39.524 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:39 smithi173 ceph-mon[30864]: osdmap e135: 12 total, 12 up, 12 in 2024-09-13T10:35:39.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.10 dump_blocked_ops 2024-09-13T10:35:39.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:39 smithi060 ceph-mon[27375]: pgmap v377: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:39.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:39 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/685398622' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-13T10:35:39.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:39 smithi060 ceph-mon[27375]: osdmap e135: 12 total, 12 up, 12 in 2024-09-13T10:35:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:39 smithi138 ceph-mon[31033]: pgmap v377: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:39 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/685398622' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-13T10:35:39.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:39 smithi138 ceph-mon[31033]: osdmap e135: 12 total, 12 up, 12 in 2024-09-13T10:35:39.770 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T10:35:40.306 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:40.306 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:40.306 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-13T10:35:40.307 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-13T10:35:40.307 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:40.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:40 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1964272253' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-13T10:35:40.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:40 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1964272253' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-13T10:35:40.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:40 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1964272253' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-13T10:35:40.934 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.10 dump_historic_ops 2024-09-13T10:35:40.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 5 1 2024-09-13T10:35:41.180 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:41.201 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:41.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:41 smithi173 ceph-mon[30864]: pgmap v379: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:41.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:41 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1964272253' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-13T10:35:41.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:41 smithi173 ceph-mon[30864]: osdmap e136: 12 total, 12 up, 12 in 2024-09-13T10:35:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:41 smithi138 ceph-mon[31033]: pgmap v379: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:41 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1964272253' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-13T10:35:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:41 smithi138 ceph-mon[31033]: osdmap e136: 12 total, 12 up, 12 in 2024-09-13T10:35:41.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:41 smithi060 ceph-mon[27375]: pgmap v379: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:41.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:41 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1964272253' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-13T10:35:41.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:41 smithi060 ceph-mon[27375]: osdmap e136: 12 total, 12 up, 12 in 2024-09-13T10:35:41.663 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:41.664 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-13T10:35:41.664 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-13T10:35:41.664 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-13T10:35:41.664 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.664 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.7 epoch 98/97 rep_tid 0 entries 98'1 (0'0) error 2:e5291ff9:::image6:head by client.24508.0:57 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-13T10:35:41.664 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.173660+0000", 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.487499155999998, 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00054537699999999999, 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.665 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173660+0000", 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173660+0000", 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.666 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173660+0000", 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.96e-07 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173688+0000", 2024-09-13T10:35:41.667 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8483000000000002e-05 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173774+0000", 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5391999999999998e-05 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.668 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173812+0000", 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7935000000000002e-05 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.173975+0000", 2024-09-13T10:35:41.669 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000163494 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.174205+0000", 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00022977699999999999 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.670 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.7 epoch 98/97 rep_tid 1 entries 98'2 (0'0) error 2:e5291ff9:::image6:head by client.24508.0:60 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.175310+0000", 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.485848947999997, 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000431116, 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.671 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175310+0000", 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.672 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175310+0000", 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175310+0000", 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4099999999999999e-07 2024-09-13T10:35:41.673 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175315+0000", 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7779999999999998e-06 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.674 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175336+0000", 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0911e-05 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175348+0000", 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2114e-05 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.675 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175397+0000", 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8531e-05 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.175741+0000", 2024-09-13T10:35:41.676 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034434100000000002 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.7 epoch 98/97 rep_tid 2 entries 98'3 (0'0) error 2:e5291ff9:::image6:head by client.24508.0:61 0.000000 -22 [r=-22+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 98'1)", 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.176284+0000", 2024-09-13T10:35:41.677 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.484875165000005, 2024-09-13T10:35:41.678 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00054298900000000002, 2024-09-13T10:35:41.678 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.678 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.678 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:41.678 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.678 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176284+0000", 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.679 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176284+0000", 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176285+0000", 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2470000000000001e-06 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.680 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.681 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176300+0000", 2024-09-13T10:35:41.684 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5349000000000002e-05 2024-09-13T10:35:41.684 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.684 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.684 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176361+0000", 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.0395e-05 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176387+0000", 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6353e-05 2024-09-13T10:35:41.685 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176529+0000", 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014238099999999999 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.686 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.176827+0000", 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029726399999999999 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.687 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:65 2.7 e98/97 2:e5291ff9:::image6:head v 98'6, mlcod=98'4)", 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.179139+0000", 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.482020203999994, 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031810900000000001, 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.688 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179139+0000", 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179139+0000", 2024-09-13T10:35:41.689 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179139+0000", 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6800000000000001e-07 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.690 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.691 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179144+0000", 2024-09-13T10:35:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3719999999999998e-06 2024-09-13T10:35:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.692 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179160+0000", 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6855e-05 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179165+0000", 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6099999999999999e-06 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.693 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179193+0000", 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8472000000000001e-05 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179205+0000", 2024-09-13T10:35:41.694 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1372999999999999e-05 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179428+0000", 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000223482 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.695 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.179457+0000", 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8476999999999999e-05 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.696 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:78 2.7 e98/97 2:e5291ff9:::image6:head v 98'13, mlcod=98'11)", 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.186289+0000", 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.474869505000001, 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031686000000000003, 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.697 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186289+0000", 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186289+0000", 2024-09-13T10:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186290+0000", 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2600000000000002e-07 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.699 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186294+0000", 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4700000000000004e-06 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186319+0000", 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.48e-05 2024-09-13T10:35:41.700 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186324+0000", 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6920000000000001e-06 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186357+0000", 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3411000000000002e-05 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186367+0000", 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5699999999999999e-06 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.702 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186582+0000", 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021513899999999999 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.186606+0000", 2024-09-13T10:35:41.703 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4151999999999998e-05 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.7 epoch 98/97 rep_tid 13 entries 98'14 (0'0) error 2:e5291ff9:::image6:head by client.24508.0:84 0.000000 -116 [r=-116+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 98'12)", 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.188463+0000", 2024-09-13T10:35:41.704 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.472695603999995, 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030124399999999999, 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188463+0000", 2024-09-13T10:35:41.705 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188463+0000", 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.706 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188464+0000", 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7999999999999996e-07 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188470+0000", 2024-09-13T10:35:41.707 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9880000000000001e-06 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188493+0000", 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2872000000000001e-05 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.708 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188501+0000", 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.5660000000000007e-06 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188557+0000", 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5324000000000001e-05 2024-09-13T10:35:41.709 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.188765+0000", 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020801399999999999 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.710 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:89 2.7 e98/97 2:e5291ff9:::image6:head v 98'17, mlcod=98'15)", 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.191377+0000", 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.469782348999999, 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030626299999999998, 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.711 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191377+0000", 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.712 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191377+0000", 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191377+0000", 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5400000000000002e-07 2024-09-13T10:35:41.713 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191390+0000", 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2661e-05 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191406+0000", 2024-09-13T10:35:41.714 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6667000000000002e-05 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191410+0000", 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8739999999999997e-06 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.715 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191440+0000", 2024-09-13T10:35:41.716 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9912000000000001e-05 2024-09-13T10:35:41.716 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.716 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.716 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.716 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191451+0000", 2024-09-13T10:35:41.717 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0842999999999999e-05 2024-09-13T10:35:41.717 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.717 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.717 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191657+0000", 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000206398 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.191683+0000", 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5454000000000001e-05 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:92 2.7 e98/97 2:e5291ff9:::image6:head v 98'18, mlcod=98'16)", 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.192759+0000", 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.468400153000005, 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035178899999999998, 2024-09-13T10:35:41.719 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192759+0000", 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192759+0000", 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.721 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192759+0000", 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2300000000000001e-07 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192780+0000", 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.0696999999999999e-05 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.722 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192800+0000", 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9735999999999999e-05 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192814+0000", 2024-09-13T10:35:41.723 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4371e-05 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192854+0000", 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0181e-05 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.724 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.192866+0000", 2024-09-13T10:35:41.725 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1936999999999999e-05 2024-09-13T10:35:41.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.725 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.725 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.725 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193079+0000", 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000212439 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193111+0000", 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1905e-05 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.726 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:93 2.7 e98/97 2:e5291ff9:::image6:head v 98'19, mlcod=98'17)", 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.193596+0000", 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.467563049999995, 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029001100000000001, 2024-09-13T10:35:41.727 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193596+0000", 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193596+0000", 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193596+0000", 2024-09-13T10:35:41.729 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8400000000000005e-07 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193599+0000", 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9960000000000002e-06 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.730 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193615+0000", 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5593000000000001e-05 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193619+0000", 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8539999999999999e-06 2024-09-13T10:35:41.731 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193648+0000", 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9561e-05 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.732 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193660+0000", 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2108e-05 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193861+0000", 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020039499999999999 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.733 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.193886+0000", 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5020000000000001e-05 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.734 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:100 2.7 e98/97 2:e5291ff9:::image6:head v 98'20, mlcod=98'18)", 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.196181+0000", 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.464977669000007, 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00034168300000000001, 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.735 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196181+0000", 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196181+0000", 2024-09-13T10:35:41.736 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196182+0000", 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.08e-07 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.737 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196185+0000", 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8849999999999999e-06 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196217+0000", 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2926999999999998e-05 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.738 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196221+0000", 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.6969999999999999e-06 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196260+0000", 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8534000000000001e-05 2024-09-13T10:35:41.739 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196273+0000", 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3166999999999999e-05 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.740 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196493+0000", 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000220063 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.196523+0000", 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0001999999999998e-05 2024-09-13T10:35:41.741 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:102 2.7 e98/97 2:e5291ff9:::image6:head v 98'21, mlcod=98'19)", 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.197454+0000", 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.463704738000004, 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000333109, 2024-09-13T10:35:41.742 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197454+0000", 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.743 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197454+0000", 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197455+0000", 2024-09-13T10:35:41.744 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3200000000000005e-07 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197468+0000", 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3030999999999999e-05 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.745 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197489+0000", 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1698000000000001e-05 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197494+0000", 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.729e-06 2024-09-13T10:35:41.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197540+0000", 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6063999999999998e-05 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197552+0000", 2024-09-13T10:35:41.747 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1943000000000001e-05 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197761+0000", 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020836800000000001 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.748 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.197787+0000", 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6644000000000001e-05 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.749 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:104 2.7 e98/97 2:e5291ff9:::image6:head v 98'23, mlcod=98'21)", 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.199118+0000", 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.462040879, 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000298793, 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.750 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199118+0000", 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199118+0000", 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199119+0000", 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.2699999999999999e-07 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.752 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199122+0000", 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.715e-06 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199140+0000", 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7827999999999999e-05 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.753 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199144+0000", 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.121e-06 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199175+0000", 2024-09-13T10:35:41.754 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0713999999999998e-05 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199185+0000", 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0373999999999999e-05 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.755 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199391+0000", 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000205324 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.199417+0000", 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.6190000000000002e-05 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.756 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:126 2.12 2:48e4d0dc:::image13:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:34.227982+0000", 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 67.433176906, 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0011521859999999999, 2024-09-13T10:35:41.757 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 126 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.758 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.227982+0000", 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.227982+0000", 2024-09-13T10:35:41.759 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.227985+0000", 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9859999999999999e-06 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.760 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.227996+0000", 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1229999999999999e-05 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.228019+0000", 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3166000000000001e-05 2024-09-13T10:35:41.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.228033+0000", 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3890000000000001e-05 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.762 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.228128+0000", 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5241000000000005e-05 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.228212+0000", 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3052000000000004e-05 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.763 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3", 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.228504+0000", 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029290799999999998 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.228826+0000", 2024-09-13T10:35:41.764 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032166300000000003 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.229077+0000", 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00025082400000000002 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.765 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.229106+0000", 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9386999999999998e-05 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:34.229134+0000", 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7838999999999999e-05 2024-09-13T10:35:41.766 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:176 2.17 e99/97 2:e9c6360c:::image20:head v 99'1, mlcod=0'0)", 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.267588+0000", 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.393570843999996, 2024-09-13T10:35:41.767 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00049514199999999996, 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267588+0000", 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267588+0000", 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.769 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267590+0000", 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8899999999999999e-06 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267605+0000", 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5342999999999999e-05 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.770 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267687+0000", 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2189999999999997e-05 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267694+0000", 2024-09-13T10:35:41.771 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.7070000000000003e-06 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267750+0000", 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6292999999999998e-05 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.772 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.267764+0000", 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3964999999999999e-05 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.268048+0000", 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00028361599999999999 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.773 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.268083+0000", 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5138000000000001e-05 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.774 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:178 2.17 e99/97 2:e9c6360c:::image20:head v 99'2, mlcod=0'0)", 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.269073+0000", 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.392086211999995, 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000436804, 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.775 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269073+0000", 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.776 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269073+0000", 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269073+0000", 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3300000000000002e-07 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.777 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.778 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269078+0000", 2024-09-13T10:35:41.778 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.775e-06 2024-09-13T10:35:41.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.778 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.778 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269104+0000", 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5588999999999998e-05 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269110+0000", 2024-09-13T10:35:41.779 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2990000000000002e-06 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269160+0000", 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.9601999999999997e-05 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.780 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269179+0000", 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9466999999999998e-05 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269477+0000", 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029805400000000002 2024-09-13T10:35:41.781 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.269509+0000", 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.2385000000000002e-05 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:180 2.17 e99/97 2:e9c6360c:::image20:head v 99'3, mlcod=99'1)", 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.270530+0000", 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.390629343000001, 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00035575400000000001, 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.783 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270530+0000", 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.784 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270530+0000", 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270530+0000", 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.89e-07 2024-09-13T10:35:41.785 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270534+0000", 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3229999999999999e-06 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270552+0000", 2024-09-13T10:35:41.786 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8068e-05 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270557+0000", 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3699999999999997e-06 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.787 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270595+0000", 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7944999999999997e-05 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270613+0000", 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7805999999999999e-05 2024-09-13T10:35:41.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270861+0000", 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00024854400000000001 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.789 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.270885+0000", 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:181 2.17 e99/97 2:e9c6360c:::image20:head v 99'4, mlcod=99'2)", 2024-09-13T10:35:41.790 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.271566+0000", 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.389592562999994, 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00032445800000000002, 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.791 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271566+0000", 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271566+0000", 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.792 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271567+0000", 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8699999999999995e-07 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.793 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271571+0000", 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6489999999999999e-06 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271588+0000", 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6167999999999998e-05 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.794 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271592+0000", 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5029999999999999e-06 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271631+0000", 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.9048999999999998e-05 2024-09-13T10:35:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271647+0000", 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.6123000000000001e-05 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.796 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271859+0000", 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021219500000000001 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.271891+0000", 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1284000000000001e-05 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.797 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing(2.17 epoch 99/97 rep_tid 30 entries 99'5 (0'0) error 2:e9c6360c:::image20:head by client.24508.0:182 0.000000 -122 [r=-122+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 99'3)", 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.272516+0000", 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.388643096999999, 2024-09-13T10:35:41.798 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029843300000000002, 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272516+0000", 2024-09-13T10:35:41.799 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272516+0000", 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.800 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272516+0000", 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9200000000000001e-07 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272523+0000", 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9009999999999997e-06 2024-09-13T10:35:41.801 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272553+0000", 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9274000000000001e-05 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272561+0000", 2024-09-13T10:35:41.802 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.6410000000000008e-06 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272626+0000", 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4503000000000002e-05 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.803 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.272814+0000", 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000188522 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.804 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:183 2.17 e99/97 2:e9c6360c:::image20:head v 99'6, mlcod=99'4)", 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.273432+0000", 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.387726674000007, 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00030070200000000002, 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.805 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273432+0000", 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273432+0000", 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.806 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273433+0000", 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.82e-07 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273437+0000", 2024-09-13T10:35:41.807 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2629999999999997e-06 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273456+0000", 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.8743e-05 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.808 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273460+0000", 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.3789999999999999e-06 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273492+0000", 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1922999999999999e-05 2024-09-13T10:35:41.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273505+0000", 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3026e-05 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.810 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273709+0000", 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00020394 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.273733+0000", 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3845999999999998e-05 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.811 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop(client.24508.0:184 2.17 e99/97 2:e9c6360c:::image20:head v 99'7, mlcod=99'5)", 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.274356+0000", 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 66.386802435999996, 2024-09-13T10:35:41.812 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000349697, 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274356+0000", 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274356+0000", 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274357+0000", 2024-09-13T10:35:41.814 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7199999999999999e-07 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274361+0000", 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2599999999999999e-06 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:41.815 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274381+0000", 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9522e-05 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274395+0000", 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4238e-05 2024-09-13T10:35:41.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274444+0000", 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.8856000000000001e-05 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:41.817 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274461+0000", 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7272e-05 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274674+0000", 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021306299999999999 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:41.818 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.274706+0000", 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1914e-05 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:41.819 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T10:35:42.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.11 dump_ops_in_flight 2024-09-13T10:35:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:42 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/400576829' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-13T10:35:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:42 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/400576829' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-13T10:35:42.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/400576829' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-13T10:35:42.681 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:42.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-13T10:35:43.164 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:43.164 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:43.164 INFO:teuthology.orchestra.run.smithi173.stdout: "num_ops": 0 2024-09-13T10:35:43.165 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:43.529 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:43 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:43.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:43 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:43.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:43 smithi173 ceph-mon[30864]: pgmap v381: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:43.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:43 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/400576829' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-13T10:35:43.530 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:43 smithi173 ceph-mon[30864]: osdmap e137: 12 total, 12 up, 12 in 2024-09-13T10:35:43.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:43 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:43.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:43 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:43.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:43 smithi060 ceph-mon[27375]: pgmap v381: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:43.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:43 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/400576829' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-13T10:35:43.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:43 smithi060 ceph-mon[27375]: osdmap e137: 12 total, 12 up, 12 in 2024-09-13T10:35:43.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:43 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:43.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:43 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:43.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:43 smithi138 ceph-mon[31033]: pgmap v381: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:43.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:43 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/400576829' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-13T10:35:43.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:43 smithi138 ceph-mon[31033]: osdmap e137: 12 total, 12 up, 12 in 2024-09-13T10:35:43.840 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.11 dump_blocked_ops 2024-09-13T10:35:44.083 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:44.275 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T10:35:44.462 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:44 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/934035900' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-13T10:35:44.560 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:44.560 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [], 2024-09-13T10:35:44.560 INFO:teuthology.orchestra.run.smithi173.stdout: "complaint_time": 30, 2024-09-13T10:35:44.561 INFO:teuthology.orchestra.run.smithi173.stdout: "num_blocked_ops": 0 2024-09-13T10:35:44.561 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:44.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:44 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/934035900' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-13T10:35:44.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:44 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/934035900' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-13T10:35:44.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-13T10:35:45.194 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:45.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph daemon osd.11 dump_historic_ops 2024-09-13T10:35:45.488 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T10:35:45.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:45 smithi173 ceph-mon[30864]: pgmap v383: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:45.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:45 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/934035900' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-13T10:35:45.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:45 smithi173 ceph-mon[30864]: osdmap e138: 12 total, 12 up, 12 in 2024-09-13T10:35:45.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:45 smithi173 ceph-mon[30864]: osdmap e139: 12 total, 12 up, 12 in 2024-09-13T10:35:45.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:45 smithi060 ceph-mon[27375]: pgmap v383: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:45.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:45 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/934035900' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-13T10:35:45.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:45 smithi060 ceph-mon[27375]: osdmap e138: 12 total, 12 up, 12 in 2024-09-13T10:35:45.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:45 smithi060 ceph-mon[27375]: osdmap e139: 12 total, 12 up, 12 in 2024-09-13T10:35:45.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:45 smithi138 ceph-mon[31033]: pgmap v383: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:45.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:45 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/934035900' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-13T10:35:45.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:45 smithi138 ceph-mon[31033]: osdmap e138: 12 total, 12 up, 12 in 2024-09-13T10:35:45.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:45 smithi138 ceph-mon[31033]: osdmap e139: 12 total, 12 up, 12 in 2024-09-13T10:35:46.036 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-09-13T10:35:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "size": 20, 2024-09-13T10:35:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 600, 2024-09-13T10:35:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "ops": [ 2024-09-13T10:35:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(1.0 86/85 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'91, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'91, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-13T10:35:46.037 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:33:33.767138+0000", 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 132.257714515, 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00077929300000000002, 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.038 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767138+0000", 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767138+0000", 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.039 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767138+0000", 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3099999999999997e-07 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767407+0000", 2024-09-13T10:35:46.040 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00026932200000000001 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767452+0000", 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4274999999999999e-05 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.041 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767471+0000", 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9624999999999999e-05 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767540+0000", 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8957999999999999e-05 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.042 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767554+0000", 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3685e-05 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.767917+0000", 2024-09-13T10:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00036279699999999999 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPush(1.0 86/85 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'92, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'92, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:33:33.768783+0000", 2024-09-13T10:35:46.044 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 132.256069008, 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00087677999999999996, 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.768783+0000", 2024-09-13T10:35:46.045 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.768783+0000", 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.046 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.768784+0000", 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.569999999999999e-07 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.769174+0000", 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000389942 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.047 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.769230+0000", 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.5792999999999999e-05 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.769260+0000", 2024-09-13T10:35:46.048 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.9804000000000001e-05 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.769335+0000", 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5251000000000002e-05 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.049 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.769342+0000", 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.9820000000000002e-06 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:33:33.769660+0000", 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031805100000000002 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.050 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(1.0,QREQUEST,e:91,from:7,reservation_nonce:2)", 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:13.001494+0000", 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.023358145000003, 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00021030500000000001, 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.051 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001494+0000", 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.052 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001494+0000", 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001495+0000", 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.207e-06 2024-09-13T10:35:46.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001501+0000", 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9839999999999998e-06 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.054 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001558+0000", 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.6598999999999998e-05 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001567+0000", 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.3489999999999998e-06 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.055 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001651+0000", 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3665000000000006e-05 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001670+0000", 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.9711000000000002e-05 2024-09-13T10:35:46.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.001704+0000", 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.379e-05 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "replica_scrub(pg: 1.0,from:0'0,to:16'92,epoch:91/85,start:1:00000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:13.002508+0000", 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.022344302999997, 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00014814200000000001, 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.058 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002508+0000", 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.059 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002508+0000", 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002509+0000", 2024-09-13T10:35:46.060 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2600000000000001e-07 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002515+0000", 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4829999999999996e-06 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.061 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002547+0000", 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1918000000000002e-05 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002555+0000", 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2810000000000008e-06 2024-09-13T10:35:46.062 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002621+0000", 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.6227000000000001e-05 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002632+0000", 2024-09-13T10:35:46.063 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.0212e-05 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.002656+0000", 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.4195000000000001e-05 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.064 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDScrubReserve(1.0,RELEASE,e:91,from:7,reservation_nonce:0)", 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:13.003961+0000", 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 93.020891335000002, 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00022100399999999999, 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.065 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.003961+0000", 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.066 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.003961+0000", 2024-09-13T10:35:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.067 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.003961+0000", 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2800000000000001e-07 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.003963+0000", 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7719999999999999e-06 2024-09-13T10:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.004026+0000", 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3429000000000005e-05 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.004031+0000", 2024-09-13T10:35:46.069 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.0919999999999997e-06 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.004153+0000", 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00012210000000000001 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.070 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.004169+0000", 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.5931e-05 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:13.004182+0000", 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2551999999999999e-05 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.071 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:152 2.a 2:57642022:::image19:head [call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.239287+0000", 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 70.785565473000005, 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0025026599999999999, 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.072 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 152 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.073 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239287+0000", 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239287+0000", 2024-09-13T10:35:46.074 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239288+0000", 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7349999999999999e-06 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.075 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239294+0000", 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9440000000000002e-06 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239305+0000", 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.114e-05 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.076 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239329+0000", 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3955e-05 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239389+0000", 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9632000000000001e-05 2024-09-13T10:35:46.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op must wait for map", 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.239395+0000", 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9950000000000002e-06 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.240514+0000", 2024-09-13T10:35:46.078 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00111929 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.240604+0000", 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9413999999999995e-05 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "waiting for subops from 3", 2024-09-13T10:35:46.079 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.240758+0000", 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000153931 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "op_commit", 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241052+0000", 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00029433999999999998 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.080 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241734+0000", 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00068192700000000003 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "commit_sent", 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241762+0000", 2024-09-13T10:35:46.081 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.8238999999999998e-05 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241789+0000", 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.7118e-05 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.082 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_repop_reply(client.24508.0:152 2.a e99/97 ondisk, result = 0)", 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.241662+0000", 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 70.783190172999994, 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000132621, 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.083 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241662+0000", 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.084 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241662+0000", 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241663+0000", 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.9500000000000001e-07 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.085 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241666+0000", 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5650000000000002e-06 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241689+0000", 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.2728999999999999e-05 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.086 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241694+0000", 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.7690000000000004e-06 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241725+0000", 2024-09-13T10:35:46.087 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.0834000000000003e-05 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241730+0000", 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7350000000000001e-06 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.088 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241795+0000", 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.4194000000000002e-05 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:153 2.a 2:57642022:::image19:head [call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:46.089 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.241870+0000", 2024-09-13T10:35:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 70.782981984000003, 2024-09-13T10:35:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015686000000000001, 2024-09-13T10:35:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.090 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 153 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.091 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241870+0000", 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241870+0000", 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.092 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241871+0000", 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.2099999999999995e-07 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241875+0000", 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.5329999999999996e-06 2024-09-13T10:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241885+0000", 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1800000000000002e-06 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241891+0000", 2024-09-13T10:35:46.094 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.9599999999999997e-06 2024-09-13T10:35:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241924+0000", 2024-09-13T10:35:46.095 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3068999999999998e-05 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.241968+0000", 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.4700000000000002e-05 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.096 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242027+0000", 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.8597000000000002e-05 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "osd_op(client.24508.0:154 2.a 2:57642022:::image19:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.242095+0000", 2024-09-13T10:35:46.097 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 70.782757377999999, 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.0053024109999999999, 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client_info": { 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client": "client.24508", 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "client_addr": "172.21.15.173:0/1369318498", 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: "tid": 154 2024-09-13T10:35:46.098 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242095+0000", 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.099 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242095+0000", 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242095+0000", 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.3200000000000004e-07 2024-09-13T10:35:46.100 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242099+0000", 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.3330000000000001e-06 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.101 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242107+0000", 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.3590000000000008e-06 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242111+0000", 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2889999999999999e-06 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.102 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242150+0000", 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.8206e-05 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.242203+0000", 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.3143000000000003e-05 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.103 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247397+0000", 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.005194349 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.104 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "pg_update_log_missing_reply(2.a epoch 99/97 rep_tid 1 lcod 99'2)", 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:34:35.247266+0000", 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 70.777585966000004, 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00013458799999999999, 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:46.105 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247266+0000", 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247266+0000", 2024-09-13T10:35:46.106 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247267+0000", 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7800000000000001e-07 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.107 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247270+0000", 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.5530000000000001e-06 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247292+0000", 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.1399e-05 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.108 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247296+0000", 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6959999999999996e-06 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247343+0000", 2024-09-13T10:35:46.109 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.6544e-05 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:34:35.247401+0000", 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 5.7818000000000001e-05 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.110 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "replica_scrub(pg: 1.0,from:0'0,to:0'0,epoch:120/119,start:1:00000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:35:11.329590+0000", 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.695262511000003, 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00018471300000000001, 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.111 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329590+0000", 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.112 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329590+0000", 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329591+0000", 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.1990000000000001e-06 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.113 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329597+0000", 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.1129999999999997e-06 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329631+0000", 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4578e-05 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.114 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329640+0000", 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9649999999999997e-06 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329736+0000", 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.5387e-05 2024-09-13T10:35:46.115 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329748+0000", 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2559e-05 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.116 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.329774+0000", 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.5911999999999999e-05 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "rep_scrubmap(1.4 e119 from shard 8)", 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:35:11.347551+0000", 2024-09-13T10:35:46.117 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 34.677300842000001, 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000126971, 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347551+0000", 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347551+0000", 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.119 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347552+0000", 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.1800000000000004e-07 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347556+0000", 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2010000000000001e-06 2024-09-13T10:35:46.120 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347594+0000", 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.7583000000000003e-05 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.121 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347601+0000", 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.396e-06 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347644+0000", 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.2979000000000003e-05 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.122 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347654+0000", 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 9.995e-06 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:11.347678+0000", 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 2.3898999999999999e-05 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.123 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "MOSDPGPull(1.c e122/121 cost 1049576)", 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:35:13.168053+0000", 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.856798658000002, 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00031386599999999999, 2024-09-13T10:35:46.124 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "reached pg", 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168053+0000", 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.125 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168053+0000", 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168054+0000", 2024-09-13T10:35:46.126 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.3099999999999997e-07 2024-09-13T10:35:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168068+0000", 2024-09-13T10:35:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.3621000000000001e-05 2024-09-13T10:35:46.127 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168130+0000", 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.2236000000000005e-05 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.128 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168137+0000", 2024-09-13T10:35:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.5070000000000003e-06 2024-09-13T10:35:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.129 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168216+0000", 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.8283999999999998e-05 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.168367+0000", 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00015158700000000001 2024-09-13T10:35:46.130 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "replica_scrub(pg: 1.13,from:0'0,to:0'0,epoch:122/121,start:1:c8000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:35:13.247626+0000", 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.777225813999998, 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.000398408, 2024-09-13T10:35:46.131 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247626+0000", 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.132 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247626+0000", 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.133 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247628+0000", 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7430000000000001e-06 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247640+0000", 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.2150000000000001e-05 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.134 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247712+0000", 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 7.2112000000000003e-05 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247730+0000", 2024-09-13T10:35:46.135 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247902+0000", 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00017259299999999999 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.136 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.247943+0000", 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.0448999999999998e-05 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.248025+0000", 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.1815000000000002e-05 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.137 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "description": "rep_scrubmap(1.b e121 from shard 0)", 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "initiated_at": "2024-09-13T10:35:13.417788+0000", 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "age": 32.607063842999999, 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0.00016126899999999999, 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "continuous": false, 2024-09-13T10:35:46.138 INFO:teuthology.orchestra.run.smithi173.stdout: "type_data": { 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "flag_point": "started", 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "events": [ 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "initiated", 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417788+0000", 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.139 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "header_read", 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417788+0000", 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 0 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "throttled", 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417789+0000", 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 8.9500000000000001e-07 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.140 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "all_read", 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417794+0000", 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 4.938e-06 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "dispatched", 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417825+0000", 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.1291000000000002e-05 2024-09-13T10:35:46.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "queued_for_pg", 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417832+0000", 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.5209999999999999e-06 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "reached_pg", 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417901+0000", 2024-09-13T10:35:46.142 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 6.8733999999999996e-05 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "started", 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417915+0000", 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 1.4790000000000001e-05 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: { 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "event": "done", 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "time": "2024-09-13T10:35:13.417949+0000", 2024-09-13T10:35:46.143 INFO:teuthology.orchestra.run.smithi173.stdout: "duration": 3.4100000000000002e-05 2024-09-13T10:35:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-09-13T10:35:46.144 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-09-13T10:35:46.144 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-09-13T10:35:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T10:35:46.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:46 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/979536100' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-13T10:35:46.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:46 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/979536100' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-13T10:35:46.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:46 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/979536100' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-13T10:35:46.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-13T10:35:47.184 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:47 smithi060 ceph-mon[27375]: pgmap v386: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:47 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/979536100' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-13T10:35:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:47 smithi060 ceph-mon[27375]: osdmap e140: 12 total, 12 up, 12 in 2024-09-13T10:35:47.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:47 smithi138 ceph-mon[31033]: pgmap v386: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:47.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:47 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/979536100' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-13T10:35:47.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:47 smithi138 ceph-mon[31033]: osdmap e140: 12 total, 12 up, 12 in 2024-09-13T10:35:47.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:47 smithi173 ceph-mon[30864]: pgmap v386: 21 pgs: 21 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:47.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:47 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/979536100' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-13T10:35:47.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:47 smithi173 ceph-mon[30864]: osdmap e140: 12 total, 12 up, 12 in 2024-09-13T10:35:48.289 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T10:35:48.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:48 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:48.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:48 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:48.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:48 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1328461722' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-13T10:35:48.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:48 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:48.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:48 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:48.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:48 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1328461722' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-13T10:35:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:48 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:48 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:48.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:48 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1328461722' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-13T10:35:48.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-13T10:35:49.182 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:49.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:49 smithi138 ceph-mon[31033]: pgmap v388: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:49.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:49 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1328461722' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-13T10:35:49.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:49 smithi138 ceph-mon[31033]: osdmap e141: 12 total, 12 up, 12 in 2024-09-13T10:35:49.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:49 smithi060 ceph-mon[27375]: pgmap v388: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:49.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:49 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1328461722' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-13T10:35:49.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:49 smithi060 ceph-mon[27375]: osdmap e141: 12 total, 12 up, 12 in 2024-09-13T10:35:49.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:49 smithi173 ceph-mon[30864]: pgmap v388: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:49.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:49 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1328461722' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-13T10:35:49.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:49 smithi173 ceph-mon[30864]: osdmap e141: 12 total, 12 up, 12 in 2024-09-13T10:35:50.298 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T10:35:50.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:50 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1526495895' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-13T10:35:50.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:50 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:50.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:50 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:50.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:50 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1526495895' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-13T10:35:50.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:50 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:50.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:50 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:50.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:50 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1526495895' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-13T10:35:50.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:50 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:50.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:50 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:51.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 8 1 2024-09-13T10:35:51.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:51 smithi060 ceph-mon[27375]: pgmap v390: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:51.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:51 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1526495895' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-13T10:35:51.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:51 smithi060 ceph-mon[27375]: osdmap e142: 12 total, 12 up, 12 in 2024-09-13T10:35:51.387 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:51.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:51 smithi138 ceph-mon[31033]: pgmap v390: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:51.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:51 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1526495895' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-13T10:35:51.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:51 smithi138 ceph-mon[31033]: osdmap e142: 12 total, 12 up, 12 in 2024-09-13T10:35:51.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:51 smithi173 ceph-mon[30864]: pgmap v390: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:51.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:51 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1526495895' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-13T10:35:51.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:51 smithi173 ceph-mon[30864]: osdmap e142: 12 total, 12 up, 12 in 2024-09-13T10:35:52.307 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.8 to 1 (10000) 2024-09-13T10:35:52.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:52 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2553893434' entity='client.admin' cmd={"prefix": "osd reweight", "id": 8, "weight": 1.0} : dispatch 2024-09-13T10:35:52.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:52 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2553893434' entity='client.admin' cmd={"prefix": "osd reweight", "id": 8, "weight": 1.0} : dispatch 2024-09-13T10:35:52.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:52 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2553893434' entity='client.admin' cmd={"prefix": "osd reweight", "id": 8, "weight": 1.0} : dispatch 2024-09-13T10:35:52.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 8 1 2024-09-13T10:35:53.235 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:53.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:53 smithi138 ceph-mon[31033]: pgmap v392: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:53.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:53 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2553893434' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]': finished 2024-09-13T10:35:53.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:53 smithi138 ceph-mon[31033]: osdmap e143: 12 total, 12 up, 12 in 2024-09-13T10:35:53.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:53 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:53.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:53 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:53.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:53 smithi060 ceph-mon[27375]: pgmap v392: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:53.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:53 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2553893434' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]': finished 2024-09-13T10:35:53.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:53 smithi060 ceph-mon[27375]: osdmap e143: 12 total, 12 up, 12 in 2024-09-13T10:35:53.615 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:53 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:53.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:53 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:53 smithi173 ceph-mon[30864]: pgmap v392: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:53 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2553893434' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 8, "weight": 1.0}]': finished 2024-09-13T10:35:53.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:53 smithi173 ceph-mon[30864]: osdmap e143: 12 total, 12 up, 12 in 2024-09-13T10:35:53.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:53 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:53.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:53 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:54.319 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-13T10:35:54.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:54 smithi138 ceph-mon[31033]: osdmap e144: 12 total, 12 up, 12 in 2024-09-13T10:35:54.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:54 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3652608071' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 8, "weight": 1.0} : dispatch 2024-09-13T10:35:54.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:54 smithi060 ceph-mon[27375]: osdmap e144: 12 total, 12 up, 12 in 2024-09-13T10:35:54.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:54 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3652608071' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 8, "weight": 1.0} : dispatch 2024-09-13T10:35:54.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:54 smithi173 ceph-mon[30864]: osdmap e144: 12 total, 12 up, 12 in 2024-09-13T10:35:54.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:54 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3652608071' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 8, "weight": 1.0} : dispatch 2024-09-13T10:35:55.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 9 1 2024-09-13T10:35:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:55.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:55 smithi138 ceph-mon[31033]: pgmap v395: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:55.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:55 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3652608071' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 8, "weight": 1.0}]': finished 2024-09-13T10:35:55.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:55 smithi138 ceph-mon[31033]: osdmap e145: 12 total, 12 up, 12 in 2024-09-13T10:35:55.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:55 smithi060 ceph-mon[27375]: pgmap v395: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:55.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:55 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3652608071' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 8, "weight": 1.0}]': finished 2024-09-13T10:35:55.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:55 smithi060 ceph-mon[27375]: osdmap e145: 12 total, 12 up, 12 in 2024-09-13T10:35:55.678 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:55 smithi173 ceph-mon[30864]: pgmap v395: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:55.679 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:55 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3652608071' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 8, "weight": 1.0}]': finished 2024-09-13T10:35:55.679 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:55 smithi173 ceph-mon[30864]: osdmap e145: 12 total, 12 up, 12 in 2024-09-13T10:35:56.327 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.9 to 1 (10000) 2024-09-13T10:35:56.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:56 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1990351524' entity='client.admin' cmd={"prefix": "osd reweight", "id": 9, "weight": 1.0} : dispatch 2024-09-13T10:35:56.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:56 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1990351524' entity='client.admin' cmd={"prefix": "osd reweight", "id": 9, "weight": 1.0} : dispatch 2024-09-13T10:35:56.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:56 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1990351524' entity='client.admin' cmd={"prefix": "osd reweight", "id": 9, "weight": 1.0} : dispatch 2024-09-13T10:35:56.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 9 1 2024-09-13T10:35:57.174 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:57.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:57 smithi060 ceph-mon[27375]: pgmap v397: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:57.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:57 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1990351524' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]': finished 2024-09-13T10:35:57.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:57 smithi060 ceph-mon[27375]: osdmap e146: 12 total, 12 up, 12 in 2024-09-13T10:35:57.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:57 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:35:57.564 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:57 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:35:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:57 smithi138 ceph-mon[31033]: pgmap v397: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:57 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1990351524' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]': finished 2024-09-13T10:35:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:57 smithi138 ceph-mon[31033]: osdmap e146: 12 total, 12 up, 12 in 2024-09-13T10:35:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:57 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:35:57.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:57 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:35:57.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:57 smithi173 ceph-mon[30864]: pgmap v397: 21 pgs: 21 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:57.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:57 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1990351524' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 9, "weight": 1.0}]': finished 2024-09-13T10:35:57.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:57 smithi173 ceph-mon[30864]: osdmap e146: 12 total, 12 up, 12 in 2024-09-13T10:35:57.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:57 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:35:57.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:57 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:35:58.337 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-13T10:35:58.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:58 smithi138 ceph-mon[31033]: osdmap e147: 12 total, 12 up, 12 in 2024-09-13T10:35:58.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:58 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3060020335' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 9, "weight": 1.0} : dispatch 2024-09-13T10:35:58.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:58 smithi060 ceph-mon[27375]: osdmap e147: 12 total, 12 up, 12 in 2024-09-13T10:35:58.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:58 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3060020335' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 9, "weight": 1.0} : dispatch 2024-09-13T10:35:58.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:58 smithi173 ceph-mon[30864]: osdmap e147: 12 total, 12 up, 12 in 2024-09-13T10:35:58.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:58 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3060020335' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 9, "weight": 1.0} : dispatch 2024-09-13T10:35:58.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 10 1 2024-09-13T10:35:59.238 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:35:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:59 smithi138 ceph-mon[31033]: pgmap v400: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:59 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3060020335' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 9, "weight": 1.0}]': finished 2024-09-13T10:35:59.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:35:59 smithi138 ceph-mon[31033]: osdmap e148: 12 total, 12 up, 12 in 2024-09-13T10:35:59.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:59 smithi060 ceph-mon[27375]: pgmap v400: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:59.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:59 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3060020335' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 9, "weight": 1.0}]': finished 2024-09-13T10:35:59.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:35:59 smithi060 ceph-mon[27375]: osdmap e148: 12 total, 12 up, 12 in 2024-09-13T10:35:59.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:59 smithi173 ceph-mon[30864]: pgmap v400: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:35:59.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:59 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3060020335' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 9, "weight": 1.0}]': finished 2024-09-13T10:35:59.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:35:59 smithi173 ceph-mon[30864]: osdmap e148: 12 total, 12 up, 12 in 2024-09-13T10:36:00.073 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.10 to 1 (10000) 2024-09-13T10:36:00.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:00 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2969057574' entity='client.admin' cmd={"prefix": "osd reweight", "id": 10, "weight": 1.0} : dispatch 2024-09-13T10:36:00.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:00 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/2969057574' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]': finished 2024-09-13T10:36:00.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:00 smithi138 ceph-mon[31033]: osdmap e149: 12 total, 12 up, 12 in 2024-09-13T10:36:00.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:00 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2969057574' entity='client.admin' cmd={"prefix": "osd reweight", "id": 10, "weight": 1.0} : dispatch 2024-09-13T10:36:00.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:00 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/2969057574' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]': finished 2024-09-13T10:36:00.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:00 smithi060 ceph-mon[27375]: osdmap e149: 12 total, 12 up, 12 in 2024-09-13T10:36:00.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 10 1 2024-09-13T10:36:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:00 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2969057574' entity='client.admin' cmd={"prefix": "osd reweight", "id": 10, "weight": 1.0} : dispatch 2024-09-13T10:36:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:00 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/2969057574' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 10, "weight": 1.0}]': finished 2024-09-13T10:36:00.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:00 smithi173 ceph-mon[30864]: osdmap e149: 12 total, 12 up, 12 in 2024-09-13T10:36:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:01.477 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:01 smithi060 ceph-mon[27375]: pgmap v402: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:01.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:01 smithi138 ceph-mon[31033]: pgmap v402: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:01.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:01 smithi173 ceph-mon[30864]: pgmap v402: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:02.351 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-13T10:36:02.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:02 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3891969627' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 10, "weight": 1.0} : dispatch 2024-09-13T10:36:02.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:02 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3891969627' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 10, "weight": 1.0} : dispatch 2024-09-13T10:36:02.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:02 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3891969627' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 10, "weight": 1.0} : dispatch 2024-09-13T10:36:03.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd reweight 11 1 2024-09-13T10:36:03.275 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:03.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:03 smithi060 ceph-mon[27375]: pgmap v404: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:03 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3891969627' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 10, "weight": 1.0}]': finished 2024-09-13T10:36:03.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:03 smithi060 ceph-mon[27375]: osdmap e150: 12 total, 12 up, 12 in 2024-09-13T10:36:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:03 smithi173 ceph-mon[30864]: pgmap v404: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:03 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3891969627' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 10, "weight": 1.0}]': finished 2024-09-13T10:36:03.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:03 smithi173 ceph-mon[30864]: osdmap e150: 12 total, 12 up, 12 in 2024-09-13T10:36:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:03 smithi138 ceph-mon[31033]: pgmap v404: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:03 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3891969627' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 10, "weight": 1.0}]': finished 2024-09-13T10:36:03.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:03 smithi138 ceph-mon[31033]: osdmap e150: 12 total, 12 up, 12 in 2024-09-13T10:36:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.11 to 1 (10000) 2024-09-13T10:36:04.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:04 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:36:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:04 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:36:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:04 smithi060 ceph-mon[27375]: osdmap e151: 12 total, 12 up, 12 in 2024-09-13T10:36:04.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:04 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/466420902' entity='client.admin' cmd={"prefix": "osd reweight", "id": 11, "weight": 1.0} : dispatch 2024-09-13T10:36:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:04 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:36:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:04 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:36:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:04 smithi173 ceph-mon[30864]: osdmap e151: 12 total, 12 up, 12 in 2024-09-13T10:36:04.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:04 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/466420902' entity='client.admin' cmd={"prefix": "osd reweight", "id": 11, "weight": 1.0} : dispatch 2024-09-13T10:36:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:04 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:36:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:04 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:36:04.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:04 smithi138 ceph-mon[31033]: osdmap e151: 12 total, 12 up, 12 in 2024-09-13T10:36:04.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:04 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/466420902' entity='client.admin' cmd={"prefix": "osd reweight", "id": 11, "weight": 1.0} : dispatch 2024-09-13T10:36:04.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 11 1 2024-09-13T10:36:05.245 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:05.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:05 smithi060 ceph-mon[27375]: pgmap v407: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:05.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:05 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/466420902' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]': finished 2024-09-13T10:36:05.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:05 smithi060 ceph-mon[27375]: osdmap e152: 12 total, 12 up, 12 in 2024-09-13T10:36:05.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:05 smithi060 ceph-mon[27375]: osdmap e153: 12 total, 12 up, 12 in 2024-09-13T10:36:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:05 smithi173 ceph-mon[30864]: pgmap v407: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:05 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/466420902' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]': finished 2024-09-13T10:36:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:05 smithi173 ceph-mon[30864]: osdmap e152: 12 total, 12 up, 12 in 2024-09-13T10:36:05.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:05 smithi173 ceph-mon[30864]: osdmap e153: 12 total, 12 up, 12 in 2024-09-13T10:36:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:05 smithi138 ceph-mon[31033]: pgmap v407: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:05.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:05 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/466420902' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 11, "weight": 1.0}]': finished 2024-09-13T10:36:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:05 smithi138 ceph-mon[31033]: osdmap e152: 12 total, 12 up, 12 in 2024-09-13T10:36:05.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:05 smithi138 ceph-mon[31033]: osdmap e153: 12 total, 12 up, 12 in 2024-09-13T10:36:06.376 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-13T10:36:06.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:06 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1132741408' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 11, "weight": 1.0} : dispatch 2024-09-13T10:36:06.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:06 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1132741408' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 11, "weight": 1.0} : dispatch 2024-09-13T10:36:06.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:06 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1132741408' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 11, "weight": 1.0} : dispatch 2024-09-13T10:36:07.045 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T10:36:07.045 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T10:36:07.045 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T10:36:07.045 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T10:36:07.046 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-13T10:36:07.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T10:36:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:07.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:07 smithi060 ceph-mon[27375]: pgmap v410: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:07.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:07 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1132741408' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 11, "weight": 1.0}]': finished 2024-09-13T10:36:07.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:07 smithi060 ceph-mon[27375]: osdmap e154: 12 total, 12 up, 12 in 2024-09-13T10:36:07.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:07 smithi173 ceph-mon[30864]: pgmap v410: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:07.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:07 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1132741408' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 11, "weight": 1.0}]': finished 2024-09-13T10:36:07.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:07 smithi173 ceph-mon[30864]: osdmap e154: 12 total, 12 up, 12 in 2024-09-13T10:36:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:07 smithi138 ceph-mon[31033]: pgmap v410: 21 pgs: 21 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:07 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1132741408' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 11, "weight": 1.0}]': finished 2024-09-13T10:36:07.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:07 smithi138 ceph-mon[31033]: osdmap e154: 12 total, 12 up, 12 in 2024-09-13T10:36:07.966 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:36:07.966 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":154,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","created":"2024-09-13T10:26:24.281921+0000","modified":"2024-09-13T10:36:06.366737+0000","last_up_change":"2024-09-13T10:33:31.317557+0000","last_in_change":"2024-09-13T10:33:11.111853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T10:29:06.355752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"119","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.2899999618530273,"score_stable":2.2899999618530273,"optimal_score":1,"raw_score_acting":2.2899999618530273,"raw_score_stable":2.2899999618530273,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3db36d02-fea0-4b18-8994-97158486c7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6803","nonce":756062006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6805","nonce":756062006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6809","nonce":756062006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6807","nonce":756062006}]},"public_addr":"172.21.15.60:6803/756062006","cluster_addr":"172.21.15.60:6805/756062006","heartbeat_back_addr":"172.21.15.60:6809/756062006","heartbeat_front_addr":"172.21.15.60:6807/756062006","state":["exists","up"]},{"osd":1,"uuid":"f2323c13-004c-47e1-81ed-44e6c9fb193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6811","nonce":3814817161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6813","nonce":3814817161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6817","nonce":3814817161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6815","nonce":3814817161}]},"public_addr":"172.21.15.60:6811/3814817161","cluster_addr":"172.21.15.60:6813/3814817161","heartbeat_back_addr":"172.21.15.60:6817/3814817161","heartbeat_front_addr":"172.21.15.60:6815/3814817161","state":["exists","up"]},{"osd":2,"uuid":"bda5f20f-8413-4a97-9969-1eea2f1f7b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6819","nonce":1235426811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6821","nonce":1235426811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6825","nonce":1235426811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6823","nonce":1235426811}]},"public_addr":"172.21.15.60:6819/1235426811","cluster_addr":"172.21.15.60:6821/1235426811","heartbeat_back_addr":"172.21.15.60:6825/1235426811","heartbeat_front_addr":"172.21.15.60:6823/1235426811","state":["exists","up"]},{"osd":3,"uuid":"43cba2dc-ab85-4d9a-9b2e-c4fd527077a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6827","nonce":3263875168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6829","nonce":3263875168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6833","nonce":3263875168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6831","nonce":3263875168}]},"public_addr":"172.21.15.60:6827/3263875168","cluster_addr":"172.21.15.60:6829/3263875168","heartbeat_back_addr":"172.21.15.60:6833/3263875168","heartbeat_front_addr":"172.21.15.60:6831/3263875168","state":["exists","up"]},{"osd":4,"uuid":"b76daf81-be0c-42b5-b262-a4a0cfd615f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6801","nonce":2143931671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2143931671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2143931671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2143931671}]},"public_addr":"172.21.15.138:6801/2143931671","cluster_addr":"172.21.15.138:6803/2143931671","heartbeat_back_addr":"172.21.15.138:6807/2143931671","heartbeat_front_addr":"172.21.15.138:6805/2143931671","state":["exists","up"]},{"osd":5,"uuid":"9037c1e7-d476-4980-8155-6244430c9011","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6809","nonce":2632797914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6811","nonce":2632797914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6815","nonce":2632797914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6813","nonce":2632797914}]},"public_addr":"172.21.15.138:6809/2632797914","cluster_addr":"172.21.15.138:6811/2632797914","heartbeat_back_addr":"172.21.15.138:6815/2632797914","heartbeat_front_addr":"172.21.15.138:6813/2632797914","state":["exists","up"]},{"osd":6,"uuid":"b635470b-4619-4a8a-b016-1811d5049544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6817","nonce":3026996312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6819","nonce":3026996312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6823","nonce":3026996312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6821","nonce":3026996312}]},"public_addr":"172.21.15.138:6817/3026996312","cluster_addr":"172.21.15.138:6819/3026996312","heartbeat_back_addr":"172.21.15.138:6823/3026996312","heartbeat_front_addr":"172.21.15.138:6821/3026996312","state":["exists","up"]},{"osd":7,"uuid":"4c58cdb7-9705-4a68-816f-8d6ddeca34d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6825","nonce":1381162744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6827","nonce":1381162744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6831","nonce":1381162744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6829","nonce":1381162744}]},"public_addr":"172.21.15.138:6825/1381162744","cluster_addr":"172.21.15.138:6827/1381162744","heartbeat_back_addr":"172.21.15.138:6831/1381162744","heartbeat_front_addr":"172.21.15.138:6829/1381162744","state":["exists","up"]},{"osd":8,"uuid":"b77d95ef-5b65-4446-ae22-ef2dfe094667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6801","nonce":1825049558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6803","nonce":1825049558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6807","nonce":1825049558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6805","nonce":1825049558}]},"public_addr":"172.21.15.173:6801/1825049558","cluster_addr":"172.21.15.173:6803/1825049558","heartbeat_back_addr":"172.21.15.173:6807/1825049558","heartbeat_front_addr":"172.21.15.173:6805/1825049558","state":["exists","up"]},{"osd":9,"uuid":"b89d58eb-56f0-4ede-aa1b-8d6c25be30af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6809","nonce":417733706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6811","nonce":417733706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6815","nonce":417733706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6813","nonce":417733706}]},"public_addr":"172.21.15.173:6809/417733706","cluster_addr":"172.21.15.173:6811/417733706","heartbeat_back_addr":"172.21.15.173:6815/417733706","heartbeat_front_addr":"172.21.15.173:6813/417733706","state":["exists","up"]},{"osd":10,"uuid":"46ddc07a-0a77-445e-bc19-a345e2cdce5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6817","nonce":3542783063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6819","nonce":3542783063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6823","nonce":3542783063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6821","nonce":3542783063}]},"public_addr":"172.21.15.173:6817/3542783063","cluster_addr":"172.21.15.173:6819/3542783063","heartbeat_back_addr":"172.21.15.173:6823/3542783063","heartbeat_front_addr":"172.21.15.173:6821/3542783063","state":["exists","up"]},{"osd":11,"uuid":"6476cc59-b51a-43a8-962e-1ae48119c362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6825","nonce":4131510497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6827","nonce":4131510497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6831","nonce":4131510497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6829","nonce":4131510497}]},"public_addr":"172.21.15.173:6825/4131510497","cluster_addr":"172.21.15.173:6827/4131510497","heartbeat_back_addr":"172.21.15.173:6831/4131510497","heartbeat_front_addr":"172.21.15.173:6829/4131510497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:34:55.434316+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:36:03.171699+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:28.640649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:52.493578+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:41.962605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:56.847894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:11.549886+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:31.793534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:49.893614+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:21.903721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:47.299595+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:34:50.226925+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/2156160235":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1488046566":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6801/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6800/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/2405347332":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1860079602":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6801/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6800/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/2669724212":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/1079154737":"2024-09-14T10:26:47.425146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:36:08.598 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:08 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1229588697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:36:08.628 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-13T10:36:08.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T10:36:08.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:08 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1229588697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:36:08.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:08 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1229588697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:36:08.882 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:36:09.534 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":154,"fsid":"74376cb6-71ba-11ef-bceb-c7b262605968","created":"2024-09-13T10:26:24.281921+0000","modified":"2024-09-13T10:36:06.366737+0000","last_up_change":"2024-09-13T10:33:31.317557+0000","last_in_change":"2024-09-13T10:33:11.111853+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T10:29:06.355752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"119","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.2899999618530273,"score_stable":2.2899999618530273,"optimal_score":1,"raw_score_acting":2.2899999618530273,"raw_score_stable":2.2899999618530273,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3db36d02-fea0-4b18-8994-97158486c7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6803","nonce":756062006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6805","nonce":756062006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6809","nonce":756062006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":756062006},{"type":"v1","addr":"172.21.15.60:6807","nonce":756062006}]},"public_addr":"172.21.15.60:6803/756062006","cluster_addr":"172.21.15.60:6805/756062006","heartbeat_back_addr":"172.21.15.60:6809/756062006","heartbeat_front_addr":"172.21.15.60:6807/756062006","state":["exists","up"]},{"osd":1,"uuid":"f2323c13-004c-47e1-81ed-44e6c9fb193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6811","nonce":3814817161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6813","nonce":3814817161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6817","nonce":3814817161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3814817161},{"type":"v1","addr":"172.21.15.60:6815","nonce":3814817161}]},"public_addr":"172.21.15.60:6811/3814817161","cluster_addr":"172.21.15.60:6813/3814817161","heartbeat_back_addr":"172.21.15.60:6817/3814817161","heartbeat_front_addr":"172.21.15.60:6815/3814817161","state":["exists","up"]},{"osd":2,"uuid":"bda5f20f-8413-4a97-9969-1eea2f1f7b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6819","nonce":1235426811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6821","nonce":1235426811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6825","nonce":1235426811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1235426811},{"type":"v1","addr":"172.21.15.60:6823","nonce":1235426811}]},"public_addr":"172.21.15.60:6819/1235426811","cluster_addr":"172.21.15.60:6821/1235426811","heartbeat_back_addr":"172.21.15.60:6825/1235426811","heartbeat_front_addr":"172.21.15.60:6823/1235426811","state":["exists","up"]},{"osd":3,"uuid":"43cba2dc-ab85-4d9a-9b2e-c4fd527077a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6827","nonce":3263875168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6829","nonce":3263875168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6833","nonce":3263875168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3263875168},{"type":"v1","addr":"172.21.15.60:6831","nonce":3263875168}]},"public_addr":"172.21.15.60:6827/3263875168","cluster_addr":"172.21.15.60:6829/3263875168","heartbeat_back_addr":"172.21.15.60:6833/3263875168","heartbeat_front_addr":"172.21.15.60:6831/3263875168","state":["exists","up"]},{"osd":4,"uuid":"b76daf81-be0c-42b5-b262-a4a0cfd615f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6801","nonce":2143931671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2143931671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2143931671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2143931671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2143931671}]},"public_addr":"172.21.15.138:6801/2143931671","cluster_addr":"172.21.15.138:6803/2143931671","heartbeat_back_addr":"172.21.15.138:6807/2143931671","heartbeat_front_addr":"172.21.15.138:6805/2143931671","state":["exists","up"]},{"osd":5,"uuid":"9037c1e7-d476-4980-8155-6244430c9011","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6809","nonce":2632797914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6811","nonce":2632797914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6815","nonce":2632797914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2632797914},{"type":"v1","addr":"172.21.15.138:6813","nonce":2632797914}]},"public_addr":"172.21.15.138:6809/2632797914","cluster_addr":"172.21.15.138:6811/2632797914","heartbeat_back_addr":"172.21.15.138:6815/2632797914","heartbeat_front_addr":"172.21.15.138:6813/2632797914","state":["exists","up"]},{"osd":6,"uuid":"b635470b-4619-4a8a-b016-1811d5049544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6817","nonce":3026996312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6819","nonce":3026996312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6823","nonce":3026996312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3026996312},{"type":"v1","addr":"172.21.15.138:6821","nonce":3026996312}]},"public_addr":"172.21.15.138:6817/3026996312","cluster_addr":"172.21.15.138:6819/3026996312","heartbeat_back_addr":"172.21.15.138:6823/3026996312","heartbeat_front_addr":"172.21.15.138:6821/3026996312","state":["exists","up"]},{"osd":7,"uuid":"4c58cdb7-9705-4a68-816f-8d6ddeca34d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6825","nonce":1381162744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6827","nonce":1381162744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6831","nonce":1381162744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1381162744},{"type":"v1","addr":"172.21.15.138:6829","nonce":1381162744}]},"public_addr":"172.21.15.138:6825/1381162744","cluster_addr":"172.21.15.138:6827/1381162744","heartbeat_back_addr":"172.21.15.138:6831/1381162744","heartbeat_front_addr":"172.21.15.138:6829/1381162744","state":["exists","up"]},{"osd":8,"uuid":"b77d95ef-5b65-4446-ae22-ef2dfe094667","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6801","nonce":1825049558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6803","nonce":1825049558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6807","nonce":1825049558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":1825049558},{"type":"v1","addr":"172.21.15.173:6805","nonce":1825049558}]},"public_addr":"172.21.15.173:6801/1825049558","cluster_addr":"172.21.15.173:6803/1825049558","heartbeat_back_addr":"172.21.15.173:6807/1825049558","heartbeat_front_addr":"172.21.15.173:6805/1825049558","state":["exists","up"]},{"osd":9,"uuid":"b89d58eb-56f0-4ede-aa1b-8d6c25be30af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6809","nonce":417733706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6811","nonce":417733706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6815","nonce":417733706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":417733706},{"type":"v1","addr":"172.21.15.173:6813","nonce":417733706}]},"public_addr":"172.21.15.173:6809/417733706","cluster_addr":"172.21.15.173:6811/417733706","heartbeat_back_addr":"172.21.15.173:6815/417733706","heartbeat_front_addr":"172.21.15.173:6813/417733706","state":["exists","up"]},{"osd":10,"uuid":"46ddc07a-0a77-445e-bc19-a345e2cdce5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6817","nonce":3542783063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6819","nonce":3542783063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6823","nonce":3542783063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3542783063},{"type":"v1","addr":"172.21.15.173:6821","nonce":3542783063}]},"public_addr":"172.21.15.173:6817/3542783063","cluster_addr":"172.21.15.173:6819/3542783063","heartbeat_back_addr":"172.21.15.173:6823/3542783063","heartbeat_front_addr":"172.21.15.173:6821/3542783063","state":["exists","up"]},{"osd":11,"uuid":"6476cc59-b51a-43a8-962e-1ae48119c362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6825","nonce":4131510497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6827","nonce":4131510497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6831","nonce":4131510497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":4131510497},{"type":"v1","addr":"172.21.15.173:6829","nonce":4131510497}]},"public_addr":"172.21.15.173:6825/4131510497","cluster_addr":"172.21.15.173:6827/4131510497","heartbeat_back_addr":"172.21.15.173:6831/4131510497","heartbeat_front_addr":"172.21.15.173:6829/4131510497","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:34:55.434316+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:36:03.171699+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:28.640649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:52.493578+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:41.962605+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:56.847894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:11.549886+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:31.793534+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:49.893614+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:21.903721+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:35:47.299595+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:34:50.226925+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/2156160235":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1488046566":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6801/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:6800/250706696":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/2405347332":"2024-09-14T10:27:11.894382+0000","172.21.15.60:0/1860079602":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6801/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:6800/3783615351":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/2669724212":"2024-09-14T10:26:47.425146+0000","172.21.15.60:0/1079154737":"2024-09-14T10:26:47.425146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:36:09.534 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:09 smithi060 ceph-mon[27375]: pgmap v412: 21 pgs: 21 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:09.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:09 smithi173 ceph-mon[30864]: pgmap v412: 21 pgs: 21 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:09.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:09 smithi138 ceph-mon[31033]: pgmap v412: 21 pgs: 21 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:10.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-13T10:36:10.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-13T10:36:10.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-13T10:36:10.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-13T10:36:10.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-13T10:36:10.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-13T10:36:10.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-13T10:36:10.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-13T10:36:10.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-13T10:36:10.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-13T10:36:10.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-13T10:36:10.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-13T10:36:10.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:10 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3277225414' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:36:10.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:10 smithi060 ceph-mon[27375]: osdmap e155: 12 total, 12 up, 12 in 2024-09-13T10:36:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:10 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3277225414' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:36:10.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:10 smithi173 ceph-mon[30864]: osdmap e155: 12 total, 12 up, 12 in 2024-09-13T10:36:10.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:10 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3277225414' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-13T10:36:10.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:10 smithi138 ceph-mon[31033]: osdmap e155: 12 total, 12 up, 12 in 2024-09-13T10:36:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:11.004 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:11.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:11 smithi060 ceph-mon[27375]: pgmap v413: 21 pgs: 21 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:11.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:11 smithi173 ceph-mon[30864]: pgmap v413: 21 pgs: 21 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:11.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:11 smithi138 ceph-mon[31033]: pgmap v413: 21 pgs: 21 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:13.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:13 smithi060 ceph-mon[27375]: pgmap v415: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:13.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:13 smithi173 ceph-mon[30864]: pgmap v415: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:13.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:13 smithi138 ceph-mon[31033]: pgmap v415: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:13.930 INFO:teuthology.orchestra.run.smithi060.stdout:223338299450 2024-09-13T10:36:13.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-13T10:36:14.684 INFO:teuthology.orchestra.run.smithi060.stdout:34359738464 2024-09-13T10:36:14.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-13T10:36:15.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:15 smithi060 ceph-mon[27375]: pgmap v416: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:15.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:15 smithi060 ceph-mon[27375]: 1.2 scrub starts 2024-09-13T10:36:15.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:15 smithi060 ceph-mon[27375]: 1.2 scrub ok 2024-09-13T10:36:15.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:15 smithi173 ceph-mon[30864]: pgmap v416: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:15.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:15 smithi173 ceph-mon[30864]: 1.2 scrub starts 2024-09-13T10:36:15.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:15 smithi173 ceph-mon[30864]: 1.2 scrub ok 2024-09-13T10:36:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:15 smithi138 ceph-mon[31033]: pgmap v416: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:15 smithi138 ceph-mon[31033]: 1.2 scrub starts 2024-09-13T10:36:15.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:15 smithi138 ceph-mon[31033]: 1.2 scrub ok 2024-09-13T10:36:16.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:16 smithi060 ceph-mon[27375]: 1.c scrub starts 2024-09-13T10:36:16.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:16 smithi060 ceph-mon[27375]: 1.c scrub ok 2024-09-13T10:36:16.727 INFO:teuthology.orchestra.run.smithi060.stdout:167503724612 2024-09-13T10:36:16.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-13T10:36:16.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:16 smithi173 ceph-mon[30864]: 1.c scrub starts 2024-09-13T10:36:16.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:16 smithi173 ceph-mon[30864]: 1.c scrub ok 2024-09-13T10:36:16.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:16 smithi138 ceph-mon[31033]: 1.c scrub starts 2024-09-13T10:36:16.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:16 smithi138 ceph-mon[31033]: 1.c scrub ok 2024-09-13T10:36:17.174 INFO:teuthology.orchestra.run.smithi060.stdout:141733920843 2024-09-13T10:36:17.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-13T10:36:17.672 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:17 smithi060 ceph-mon[27375]: pgmap v417: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:17.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:17 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:36:17.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:17 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:36:17.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:17 smithi060 ceph-mon[27375]: osdmap e156: 12 total, 12 up, 12 in 2024-09-13T10:36:17.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:17 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:36:17.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:17 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:36:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:17 smithi173 ceph-mon[30864]: pgmap v417: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:17 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:36:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:17 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:36:17.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:17 smithi173 ceph-mon[30864]: osdmap e156: 12 total, 12 up, 12 in 2024-09-13T10:36:17.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:17 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:36:17.778 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:17 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:36:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:17 smithi138 ceph-mon[31033]: pgmap v417: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:17 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:36:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:17 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:36:17.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:17 smithi138 ceph-mon[31033]: osdmap e156: 12 total, 12 up, 12 in 2024-09-13T10:36:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:17 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:36:17.851 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:17 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:36:18.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:18 smithi060 ceph-mon[27375]: 1.5 scrub starts 2024-09-13T10:36:18.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:18 smithi060 ceph-mon[27375]: 1.5 scrub ok 2024-09-13T10:36:18.690 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:18 smithi060 ceph-mon[27375]: osdmap e157: 12 total, 12 up, 12 in 2024-09-13T10:36:18.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:18 smithi173 ceph-mon[30864]: 1.5 scrub starts 2024-09-13T10:36:18.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:18 smithi173 ceph-mon[30864]: 1.5 scrub ok 2024-09-13T10:36:18.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:18 smithi173 ceph-mon[30864]: osdmap e157: 12 total, 12 up, 12 in 2024-09-13T10:36:18.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:18 smithi138 ceph-mon[31033]: 1.5 scrub starts 2024-09-13T10:36:18.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:18 smithi138 ceph-mon[31033]: 1.5 scrub ok 2024-09-13T10:36:18.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:18 smithi138 ceph-mon[31033]: osdmap e157: 12 total, 12 up, 12 in 2024-09-13T10:36:19.111 INFO:teuthology.orchestra.run.smithi060.stdout:360777252899 2024-09-13T10:36:19.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-13T10:36:19.561 INFO:teuthology.orchestra.run.smithi060.stdout:90194313299 2024-09-13T10:36:19.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-13T10:36:19.622 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:19 smithi060 ceph-mon[27375]: pgmap v420: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:19.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:19 smithi060 ceph-mon[27375]: 1.b scrub starts 2024-09-13T10:36:19.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:19 smithi060 ceph-mon[27375]: 1.b scrub ok 2024-09-13T10:36:19.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:19 smithi173 ceph-mon[30864]: pgmap v420: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:19.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:19 smithi173 ceph-mon[30864]: 1.b scrub starts 2024-09-13T10:36:19.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:19 smithi173 ceph-mon[30864]: 1.b scrub ok 2024-09-13T10:36:19.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:19 smithi138 ceph-mon[31033]: pgmap v420: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:19.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:19 smithi138 ceph-mon[31033]: 1.b scrub starts 2024-09-13T10:36:19.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:19 smithi138 ceph-mon[31033]: 1.b scrub ok 2024-09-13T10:36:20.929 INFO:teuthology.orchestra.run.smithi060.stdout:55834574940 2024-09-13T10:36:20.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-13T10:36:21.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:21 smithi060 ceph-mon[27375]: 1.a scrub starts 2024-09-13T10:36:21.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:21 smithi060 ceph-mon[27375]: 1.a scrub ok 2024-09-13T10:36:21.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:21 smithi060 ceph-mon[27375]: pgmap v421: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:21.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:21 smithi060 ceph-mon[27375]: 1.6 scrub starts 2024-09-13T10:36:21.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:21 smithi060 ceph-mon[27375]: 1.6 scrub ok 2024-09-13T10:36:21.299 INFO:teuthology.orchestra.run.smithi060.stdout:115964117071 2024-09-13T10:36:21.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-13T10:36:21.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:21 smithi138 ceph-mon[31033]: 1.a scrub starts 2024-09-13T10:36:21.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:21 smithi138 ceph-mon[31033]: 1.a scrub ok 2024-09-13T10:36:21.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:21 smithi138 ceph-mon[31033]: pgmap v421: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:21.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:21 smithi138 ceph-mon[31033]: 1.6 scrub starts 2024-09-13T10:36:21.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:21 smithi138 ceph-mon[31033]: 1.6 scrub ok 2024-09-13T10:36:21.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:21 smithi173 ceph-mon[30864]: 1.a scrub starts 2024-09-13T10:36:21.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:21 smithi173 ceph-mon[30864]: 1.a scrub ok 2024-09-13T10:36:21.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:21 smithi173 ceph-mon[30864]: pgmap v421: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:21.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:21 smithi173 ceph-mon[30864]: 1.6 scrub starts 2024-09-13T10:36:21.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:21 smithi173 ceph-mon[30864]: 1.6 scrub ok 2024-09-13T10:36:22.692 INFO:teuthology.orchestra.run.smithi060.stdout:261993005110 2024-09-13T10:36:22.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-13T10:36:22.924 INFO:teuthology.orchestra.run.smithi060.stdout:296352743473 2024-09-13T10:36:22.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-13T10:36:23.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: 1.8 scrub starts 2024-09-13T10:36:23.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: 1.8 scrub ok 2024-09-13T10:36:23.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: pgmap v422: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:23.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: 1.9 scrub starts 2024-09-13T10:36:23.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: 1.9 scrub ok 2024-09-13T10:36:23.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: 1.e scrub starts 2024-09-13T10:36:23.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:23 smithi060 ceph-mon[27375]: 1.e scrub ok 2024-09-13T10:36:23.304 INFO:teuthology.orchestra.run.smithi060.stdout:330712481834 2024-09-13T10:36:23.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-13T10:36:23.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: 1.8 scrub starts 2024-09-13T10:36:23.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: 1.8 scrub ok 2024-09-13T10:36:23.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: pgmap v422: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:23.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: 1.9 scrub starts 2024-09-13T10:36:23.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: 1.9 scrub ok 2024-09-13T10:36:23.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: 1.e scrub starts 2024-09-13T10:36:23.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:23 smithi138 ceph-mon[31033]: 1.e scrub ok 2024-09-13T10:36:23.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: 1.8 scrub starts 2024-09-13T10:36:23.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: 1.8 scrub ok 2024-09-13T10:36:23.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: pgmap v422: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:23.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: 1.9 scrub starts 2024-09-13T10:36:23.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: 1.9 scrub ok 2024-09-13T10:36:23.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: 1.e scrub starts 2024-09-13T10:36:23.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:23 smithi173 ceph-mon[30864]: 1.e scrub ok 2024-09-13T10:36:24.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:24 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:36:24.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:24 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:36:24.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:24 smithi138 ceph-mon[31033]: osdmap e158: 12 total, 12 up, 12 in 2024-09-13T10:36:24.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:24 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:36:24.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:24 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:36:24.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:24 smithi060 ceph-mon[27375]: osdmap e158: 12 total, 12 up, 12 in 2024-09-13T10:36:24.468 INFO:teuthology.orchestra.run.smithi060.stdout:197568495681 2024-09-13T10:36:24.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-13T10:36:24.514 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.522 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:24 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:36:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:24 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:36:24.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:24 smithi173 ceph-mon[30864]: osdmap e158: 12 total, 12 up, 12 in 2024-09-13T10:36:24.535 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.535 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.588 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.592 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:24.618 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:25.248 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:25 smithi060 ceph-mon[27375]: pgmap v424: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:25.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:25 smithi138 ceph-mon[31033]: pgmap v424: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:25.428 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:25 smithi173 ceph-mon[30864]: pgmap v424: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:25.536 INFO:teuthology.orchestra.run.smithi060.stdout:167503724614 2024-09-13T10:36:25.722 INFO:teuthology.orchestra.run.smithi060.stdout:360777252901 2024-09-13T10:36:26.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:26 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3972463941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T10:36:26.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:26 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/867108526' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-13T10:36:26.507 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:26 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3972463941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T10:36:26.507 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:26 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/867108526' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-13T10:36:26.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:26 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3972463941' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-13T10:36:26.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:26 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/867108526' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-13T10:36:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:27.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 360777252899 got 360777252901 for osd.11 2024-09-13T10:36:27.141 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:27.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:27 smithi060 ceph-mon[27375]: pgmap v425: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:27.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:27 smithi138 ceph-mon[31033]: pgmap v425: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:27.459 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:27.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:27 smithi173 ceph-mon[30864]: pgmap v425: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:28.462 INFO:teuthology.orchestra.run.smithi060.stdout:223338299453 2024-09-13T10:36:28.654 INFO:teuthology.orchestra.run.smithi060.stdout:261993005112 2024-09-13T10:36:29.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:29 smithi060 ceph-mon[27375]: pgmap v426: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:29.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:29 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/4217016436' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T10:36:29.301 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:29 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/33910858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T10:36:29.302 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:29 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3972160399' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-13T10:36:29.441 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724612 got 167503724614 for osd.5 2024-09-13T10:36:29.442 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:29.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:29 smithi173 ceph-mon[30864]: pgmap v426: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:29.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:29 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/4217016436' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T10:36:29.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:29 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/33910858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T10:36:29.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:29 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3972160399' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-13T10:36:29.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:29 smithi138 ceph-mon[31033]: pgmap v426: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:29.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:29 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/4217016436' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-13T10:36:29.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:29 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/33910858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-13T10:36:29.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:29 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3972160399' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-13T10:36:30.089 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005110 got 261993005112 for osd.8 2024-09-13T10:36:30.089 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:30.209 INFO:teuthology.orchestra.run.smithi060.stdout:34359738467 2024-09-13T10:36:30.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:30 smithi060 ceph-mon[27375]: 1.1 scrub starts 2024-09-13T10:36:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:30 smithi060 ceph-mon[27375]: 1.1 scrub ok 2024-09-13T10:36:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:30 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3772736375' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-13T10:36:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:30 smithi060 ceph-mon[27375]: 1.7 scrub starts 2024-09-13T10:36:30.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:30 smithi060 ceph-mon[27375]: 1.7 scrub ok 2024-09-13T10:36:30.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:30 smithi173 ceph-mon[30864]: 1.1 scrub starts 2024-09-13T10:36:30.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:30 smithi173 ceph-mon[30864]: 1.1 scrub ok 2024-09-13T10:36:30.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:30 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3772736375' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-13T10:36:30.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:30 smithi173 ceph-mon[30864]: 1.7 scrub starts 2024-09-13T10:36:30.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:30 smithi173 ceph-mon[30864]: 1.7 scrub ok 2024-09-13T10:36:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:30 smithi138 ceph-mon[31033]: 1.1 scrub starts 2024-09-13T10:36:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:30 smithi138 ceph-mon[31033]: 1.1 scrub ok 2024-09-13T10:36:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:30 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3772736375' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-13T10:36:30.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:30 smithi138 ceph-mon[31033]: 1.7 scrub starts 2024-09-13T10:36:30.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:30 smithi138 ceph-mon[31033]: 1.7 scrub ok 2024-09-13T10:36:30.812 INFO:teuthology.orchestra.run.smithi060.stdout:55834574942 2024-09-13T10:36:31.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:31 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:36:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:31 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:36:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:31 smithi060 ceph-mon[27375]: pgmap v427: 21 pgs: 21 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:31 smithi060 ceph-mon[27375]: osdmap e159: 12 total, 12 up, 12 in 2024-09-13T10:36:31.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:31 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1286737031' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T10:36:31.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574940 got 55834574942 for osd.1 2024-09-13T10:36:31.431 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:31 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:36:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:31 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:36:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:31 smithi173 ceph-mon[30864]: pgmap v427: 21 pgs: 21 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:31.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:31 smithi173 ceph-mon[30864]: osdmap e159: 12 total, 12 up, 12 in 2024-09-13T10:36:31.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:31 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1286737031' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T10:36:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:31 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:36:31.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:31 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:36:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:31 smithi138 ceph-mon[31033]: pgmap v427: 21 pgs: 21 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:31 smithi138 ceph-mon[31033]: osdmap e159: 12 total, 12 up, 12 in 2024-09-13T10:36:31.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:31 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1286737031' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-13T10:36:32.207 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:32 smithi060 ceph-mon[27375]: 1.3 scrub starts 2024-09-13T10:36:32.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:32 smithi060 ceph-mon[27375]: 1.3 scrub ok 2024-09-13T10:36:32.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:32 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3383173639' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T10:36:32.389 INFO:teuthology.orchestra.run.smithi060.stdout:296352743476 2024-09-13T10:36:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:32 smithi173 ceph-mon[30864]: 1.3 scrub starts 2024-09-13T10:36:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:32 smithi173 ceph-mon[30864]: 1.3 scrub ok 2024-09-13T10:36:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:32 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3383173639' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T10:36:32.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:32 smithi138 ceph-mon[31033]: 1.3 scrub starts 2024-09-13T10:36:32.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:32 smithi138 ceph-mon[31033]: 1.3 scrub ok 2024-09-13T10:36:32.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:32 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3383173639' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-13T10:36:33.043 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743473 got 296352743476 for osd.9 2024-09-13T10:36:33.043 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:33.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:33 smithi060 ceph-mon[27375]: pgmap v429: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:33 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1333175957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-13T10:36:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:33 smithi060 ceph-mon[27375]: 1.14 scrub starts 2024-09-13T10:36:33.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:33 smithi060 ceph-mon[27375]: 1.14 scrub ok 2024-09-13T10:36:33.444 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738464 got 34359738467 for osd.0 2024-09-13T10:36:33.444 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:33 smithi173 ceph-mon[30864]: pgmap v429: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:33 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1333175957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-13T10:36:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:33 smithi173 ceph-mon[30864]: 1.14 scrub starts 2024-09-13T10:36:33.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:33 smithi173 ceph-mon[30864]: 1.14 scrub ok 2024-09-13T10:36:33.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299450 got 223338299453 for osd.7 2024-09-13T10:36:33.564 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:33.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:33 smithi138 ceph-mon[31033]: pgmap v429: 21 pgs: 21 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:33 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1333175957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-13T10:36:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:33 smithi138 ceph-mon[31033]: 1.14 scrub starts 2024-09-13T10:36:33.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:33 smithi138 ceph-mon[31033]: 1.14 scrub ok 2024-09-13T10:36:33.782 INFO:teuthology.orchestra.run.smithi060.stdout:330712481835 2024-09-13T10:36:34.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481834 got 330712481835 for osd.10 2024-09-13T10:36:34.380 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:34.746 INFO:teuthology.orchestra.run.smithi060.stdout:141733920847 2024-09-13T10:36:34.975 INFO:teuthology.orchestra.run.smithi060.stdout:90194313303 2024-09-13T10:36:35.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:35 smithi060 ceph-mon[27375]: pgmap v430: 21 pgs: 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:35.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:35 smithi060 ceph-mon[27375]: 1.13 scrub starts 2024-09-13T10:36:35.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:35 smithi060 ceph-mon[27375]: 1.13 scrub ok 2024-09-13T10:36:35.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:35 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/285771705' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T10:36:35.369 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:35 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3757141267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T10:36:35.408 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920843 got 141733920847 for osd.4 2024-09-13T10:36:35.409 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:35.502 INFO:teuthology.orchestra.run.smithi060.stdout:115964117074 2024-09-13T10:36:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:35 smithi173 ceph-mon[30864]: pgmap v430: 21 pgs: 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:35 smithi173 ceph-mon[30864]: 1.13 scrub starts 2024-09-13T10:36:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:35 smithi173 ceph-mon[30864]: 1.13 scrub ok 2024-09-13T10:36:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:35 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/285771705' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T10:36:35.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:35 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3757141267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T10:36:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:35 smithi138 ceph-mon[31033]: pgmap v430: 21 pgs: 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:35 smithi138 ceph-mon[31033]: 1.13 scrub starts 2024-09-13T10:36:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:35 smithi138 ceph-mon[31033]: 1.13 scrub ok 2024-09-13T10:36:35.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:35 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/285771705' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-13T10:36:35.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:35 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3757141267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-13T10:36:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:36.120 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117071 got 115964117074 for osd.3 2024-09-13T10:36:36.121 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:36.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:36 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:36:36.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:36 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:36:36.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:36 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:36:36.987 INFO:teuthology.orchestra.run.smithi060.stdout:197568495683 2024-09-13T10:36:37.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313299 got 90194313303 for osd.2 2024-09-13T10:36:37.259 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:37.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:37 smithi138 ceph-mon[31033]: pgmap v431: 21 pgs: 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:37.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:36:37.350 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:37 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:36:37.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:37 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:36:37.351 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:37 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/3170068684' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T10:36:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:37 smithi173 ceph-mon[30864]: pgmap v431: 21 pgs: 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:36:37.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:37 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:36:37.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:37 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:36:37.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:37 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/3170068684' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T10:36:37.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:37 smithi060 ceph-mon[27375]: pgmap v431: 21 pgs: 21 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:37.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:36:37.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:37 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:36:37.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:37 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:36:37.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:37 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/3170068684' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-13T10:36:37.929 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495681 got 197568495683 for osd.6 2024-09-13T10:36:37.929 DEBUG:teuthology.parallel:result is None 2024-09-13T10:36:37.930 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T10:36:37.930 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:36:38.195 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:38 smithi173 ceph-mon[30864]: purged_snaps scrub starts 2024-09-13T10:36:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:38 smithi173 ceph-mon[30864]: purged_snaps scrub ok 2024-09-13T10:36:38.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:38 smithi173 ceph-mon[30864]: osdmap e160: 12 total, 12 up, 12 in 2024-09-13T10:36:38.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:38 smithi060 ceph-mon[27375]: purged_snaps scrub starts 2024-09-13T10:36:38.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:38 smithi060 ceph-mon[27375]: purged_snaps scrub ok 2024-09-13T10:36:38.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:38 smithi060 ceph-mon[27375]: osdmap e160: 12 total, 12 up, 12 in 2024-09-13T10:36:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:38 smithi138 ceph-mon[31033]: purged_snaps scrub starts 2024-09-13T10:36:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:38 smithi138 ceph-mon[31033]: purged_snaps scrub ok 2024-09-13T10:36:38.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:38 smithi138 ceph-mon[31033]: osdmap e160: 12 total, 12 up, 12 in 2024-09-13T10:36:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:36:38.843 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:36:39.409 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:39 smithi060 ceph-mon[27375]: pgmap v433: 21 pgs: 21 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:39.442 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-13T10:36:38.001664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1932,"ondisk_log_size":1932,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369984,"kb_used_data":14552,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124571840,"statfs":{"total":1151940427776,"available":1151561564160,"internally_reserved":0,"allocated":14901248,"data_stored":9404359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001325"},"pg_stats":[{"pgid":"1.14","version":"16'92","reported_seq":102,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:32.551014+0000","last_change":"2024-09-13T10:36:32.551014+0000","last_active":"2024-09-13T10:36:32.551014+0000","last_peered":"2024-09-13T10:36:32.551014+0000","last_clean":"2024-09-13T10:36:32.551014+0000","last_became_active":"2024-09-13T10:35:13.165585+0000","last_became_peered":"2024-09-13T10:35:13.165585+0000","last_unstale":"2024-09-13T10:36:32.551014+0000","last_undegraded":"2024-09-13T10:36:32.551014+0000","last_fullsized":"2024-09-13T10:36:32.551014+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:32.550965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:32.550965+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:55.788595+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'92","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:34.231748+0000","last_change":"2024-09-13T10:36:34.231748+0000","last_active":"2024-09-13T10:36:34.231748+0000","last_peered":"2024-09-13T10:36:34.231748+0000","last_clean":"2024-09-13T10:36:34.231748+0000","last_became_active":"2024-09-13T10:35:13.167887+0000","last_became_peered":"2024-09-13T10:35:13.167887+0000","last_unstale":"2024-09-13T10:36:34.231748+0000","last_undegraded":"2024-09-13T10:36:34.231748+0000","last_fullsized":"2024-09-13T10:36:34.231748+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:34.231682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:34.231682+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:54.889669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11],"acting":[1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'92","reported_seq":91,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.110166+0000","last_change":"2024-09-13T10:35:14.497030+0000","last_active":"2024-09-13T10:36:30.110166+0000","last_peered":"2024-09-13T10:36:30.110166+0000","last_clean":"2024-09-13T10:36:30.110166+0000","last_became_active":"2024-09-13T10:35:13.165149+0000","last_became_peered":"2024-09-13T10:35:13.165149+0000","last_unstale":"2024-09-13T10:36:30.110166+0000","last_undegraded":"2024-09-13T10:36:30.110166+0000","last_fullsized":"2024-09-13T10:36:30.110166+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:14.496969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:14.496969+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:43.515471+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'92","reported_seq":91,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.109086+0000","last_change":"2024-09-13T10:35:14.052060+0000","last_active":"2024-09-13T10:36:30.109086+0000","last_peered":"2024-09-13T10:36:30.109086+0000","last_clean":"2024-09-13T10:36:30.109086+0000","last_became_active":"2024-09-13T10:35:13.167374+0000","last_became_peered":"2024-09-13T10:35:13.167374+0000","last_unstale":"2024-09-13T10:36:30.109086+0000","last_undegraded":"2024-09-13T10:36:30.109086+0000","last_fullsized":"2024-09-13T10:36:30.109086+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:14.052000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:14.052000+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:43.111845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"16'92","reported_seq":91,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.110127+0000","last_change":"2024-09-13T10:35:13.456223+0000","last_active":"2024-09-13T10:36:30.110127+0000","last_peered":"2024-09-13T10:36:30.110127+0000","last_clean":"2024-09-13T10:36:30.110127+0000","last_became_active":"2024-09-13T10:35:13.163670+0000","last_became_peered":"2024-09-13T10:35:13.163670+0000","last_unstale":"2024-09-13T10:36:30.110127+0000","last_undegraded":"2024-09-13T10:36:30.110127+0000","last_fullsized":"2024-09-13T10:36:30.110127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:13.456177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:13.456177+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:42.956882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'92","reported_seq":130,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.241481+0000","last_change":"2024-09-13T10:35:13.714630+0000","last_active":"2024-09-13T10:36:37.241481+0000","last_peered":"2024-09-13T10:36:37.241481+0000","last_clean":"2024-09-13T10:36:37.241481+0000","last_became_active":"2024-09-13T10:35:13.166768+0000","last_became_peered":"2024-09-13T10:35:13.166768+0000","last_unstale":"2024-09-13T10:36:37.241481+0000","last_undegraded":"2024-09-13T10:36:37.241481+0000","last_fullsized":"2024-09-13T10:36:37.241481+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:13.714577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:13.714577+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:41.321833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'92","reported_seq":102,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600945+0000","last_change":"2024-09-13T10:36:22.663444+0000","last_active":"2024-09-13T10:36:30.600945+0000","last_peered":"2024-09-13T10:36:30.600945+0000","last_clean":"2024-09-13T10:36:30.600945+0000","last_became_active":"2024-09-13T10:35:13.165018+0000","last_became_peered":"2024-09-13T10:35:13.165018+0000","last_unstale":"2024-09-13T10:36:30.600945+0000","last_undegraded":"2024-09-13T10:36:30.600945+0000","last_fullsized":"2024-09-13T10:36:30.600945+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:22.663361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:22.663361+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:28.652686+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.d","version":"16'92","reported_seq":93,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.601221+0000","last_change":"2024-09-13T10:35:13.374342+0000","last_active":"2024-09-13T10:36:30.601221+0000","last_peered":"2024-09-13T10:36:30.601221+0000","last_clean":"2024-09-13T10:36:30.601221+0000","last_became_active":"2024-09-13T10:35:13.164676+0000","last_became_peered":"2024-09-13T10:35:13.164676+0000","last_unstale":"2024-09-13T10:36:30.601221+0000","last_undegraded":"2024-09-13T10:36:30.601221+0000","last_fullsized":"2024-09-13T10:36:30.601221+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:13.374285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:13.374285+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:40.751084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"16'92","reported_seq":106,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.146022+0000","last_change":"2024-09-13T10:36:14.842038+0000","last_active":"2024-09-13T10:36:30.146022+0000","last_peered":"2024-09-13T10:36:30.146022+0000","last_clean":"2024-09-13T10:36:30.146022+0000","last_became_active":"2024-09-13T10:35:13.165970+0000","last_became_peered":"2024-09-13T10:35:13.165970+0000","last_unstale":"2024-09-13T10:36:30.146022+0000","last_undegraded":"2024-09-13T10:36:30.146022+0000","last_fullsized":"2024-09-13T10:36:30.146022+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:14.841985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:14.841985+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:16.529239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'92","reported_seq":101,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.460071+0000","last_change":"2024-09-13T10:36:18.690508+0000","last_active":"2024-09-13T10:36:30.460071+0000","last_peered":"2024-09-13T10:36:30.460071+0000","last_clean":"2024-09-13T10:36:30.460071+0000","last_became_active":"2024-09-13T10:35:13.168017+0000","last_became_peered":"2024-09-13T10:35:13.168017+0000","last_unstale":"2024-09-13T10:36:30.460071+0000","last_undegraded":"2024-09-13T10:36:30.460071+0000","last_fullsized":"2024-09-13T10:36:30.460071+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:18.690422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:18.690422+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:25.696673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.8","version":"16'92","reported_seq":130,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.110140+0000","last_change":"2024-09-13T10:36:21.812385+0000","last_active":"2024-09-13T10:36:30.110140+0000","last_peered":"2024-09-13T10:36:30.110140+0000","last_clean":"2024-09-13T10:36:30.110140+0000","last_became_active":"2024-09-13T10:35:01.488976+0000","last_became_peered":"2024-09-13T10:35:01.488976+0000","last_unstale":"2024-09-13T10:36:30.110140+0000","last_undegraded":"2024-09-13T10:36:30.110140+0000","last_fullsized":"2024-09-13T10:36:30.110140+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:21.812316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:21.812316+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:46.770333+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'92","reported_seq":302,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600945+0000","last_change":"2024-09-13T10:36:22.429031+0000","last_active":"2024-09-13T10:36:30.600945+0000","last_peered":"2024-09-13T10:36:30.600945+0000","last_clean":"2024-09-13T10:36:30.600945+0000","last_became_active":"2024-09-13T10:35:01.108329+0000","last_became_peered":"2024-09-13T10:35:01.108329+0000","last_unstale":"2024-09-13T10:36:30.600945+0000","last_undegraded":"2024-09-13T10:36:30.600945+0000","last_fullsized":"2024-09-13T10:36:30.600945+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:22.428973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:22.428973+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:44.863164+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"16'92","reported_seq":121,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600651+0000","last_change":"2024-09-13T10:36:19.993329+0000","last_active":"2024-09-13T10:36:30.600651+0000","last_peered":"2024-09-13T10:36:30.600651+0000","last_clean":"2024-09-13T10:36:30.600651+0000","last_became_active":"2024-09-13T10:35:01.107379+0000","last_became_peered":"2024-09-13T10:35:01.107379+0000","last_unstale":"2024-09-13T10:36:30.600651+0000","last_undegraded":"2024-09-13T10:36:30.600651+0000","last_fullsized":"2024-09-13T10:36:30.600651+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:19.993270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:19.993270+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:39.451549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6],"acting":[9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"16'92","reported_seq":229,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.460070+0000","last_change":"2024-09-13T10:35:11.347909+0000","last_active":"2024-09-13T10:36:30.460070+0000","last_peered":"2024-09-13T10:36:30.460070+0000","last_clean":"2024-09-13T10:36:30.460070+0000","last_became_active":"2024-09-13T10:35:11.067384+0000","last_became_peered":"2024-09-13T10:35:11.067384+0000","last_unstale":"2024-09-13T10:36:30.460070+0000","last_undegraded":"2024-09-13T10:36:30.460070+0000","last_fullsized":"2024-09-13T10:36:30.460070+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:11.347854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:11.347854+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:38.534360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8],"acting":[11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.5","version":"16'92","reported_seq":130,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600857+0000","last_change":"2024-09-13T10:36:16.503547+0000","last_active":"2024-09-13T10:36:30.600857+0000","last_peered":"2024-09-13T10:36:30.600857+0000","last_clean":"2024-09-13T10:36:30.600857+0000","last_became_active":"2024-09-13T10:35:11.070872+0000","last_became_peered":"2024-09-13T10:35:11.070872+0000","last_unstale":"2024-09-13T10:36:30.600857+0000","last_undegraded":"2024-09-13T10:36:30.600857+0000","last_fullsized":"2024-09-13T10:36:30.600857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:16.503486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:16.503486+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:17.090008+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2],"acting":[10,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.6","version":"16'92","reported_seq":132,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600861+0000","last_change":"2024-09-13T10:36:20.447174+0000","last_active":"2024-09-13T10:36:30.600861+0000","last_peered":"2024-09-13T10:36:30.600861+0000","last_clean":"2024-09-13T10:36:30.600861+0000","last_became_active":"2024-09-13T10:35:11.071406+0000","last_became_peered":"2024-09-13T10:35:11.071406+0000","last_unstale":"2024-09-13T10:36:30.600861+0000","last_undegraded":"2024-09-13T10:36:30.600861+0000","last_fullsized":"2024-09-13T10:36:30.600861+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:20.447111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:20.447111+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:21.106848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9],"acting":[10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.7","version":"16'92","reported_seq":133,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.241499+0000","last_change":"2024-09-13T10:36:29.960118+0000","last_active":"2024-09-13T10:36:37.241499+0000","last_peered":"2024-09-13T10:36:37.241499+0000","last_clean":"2024-09-13T10:36:37.241499+0000","last_became_active":"2024-09-13T10:35:11.069951+0000","last_became_peered":"2024-09-13T10:35:11.069951+0000","last_unstale":"2024-09-13T10:36:37.241499+0000","last_undegraded":"2024-09-13T10:36:37.241499+0000","last_fullsized":"2024-09-13T10:36:37.241499+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:29.960042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:29.960042+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:35.190521+0000","scrub_duration":282,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"16'92","reported_seq":130,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600857+0000","last_change":"2024-09-13T10:36:29.286435+0000","last_active":"2024-09-13T10:36:30.600857+0000","last_peered":"2024-09-13T10:36:30.600857+0000","last_clean":"2024-09-13T10:36:30.600857+0000","last_became_active":"2024-09-13T10:35:11.071470+0000","last_became_peered":"2024-09-13T10:35:11.071470+0000","last_unstale":"2024-09-13T10:36:30.600857+0000","last_undegraded":"2024-09-13T10:36:30.600857+0000","last_fullsized":"2024-09-13T10:36:30.600857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:29.286359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:29.286359+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:58.766540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9],"acting":[10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.2","version":"16'92","reported_seq":313,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.601096+0000","last_change":"2024-09-13T10:36:14.382942+0000","last_active":"2024-09-13T10:36:30.601096+0000","last_peered":"2024-09-13T10:36:30.601096+0000","last_clean":"2024-09-13T10:36:30.601096+0000","last_became_active":"2024-09-13T10:35:11.069026+0000","last_became_peered":"2024-09-13T10:35:11.069026+0000","last_unstale":"2024-09-13T10:36:30.601096+0000","last_undegraded":"2024-09-13T10:36:30.601096+0000","last_fullsized":"2024-09-13T10:36:30.601096+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:14.382852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:14.382852+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:21.634070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'92","reported_seq":133,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:31.175963+0000","last_change":"2024-09-13T10:36:31.175963+0000","last_active":"2024-09-13T10:36:31.175963+0000","last_peered":"2024-09-13T10:36:31.175963+0000","last_clean":"2024-09-13T10:36:31.175963+0000","last_became_active":"2024-09-13T10:35:11.069433+0000","last_became_peered":"2024-09-13T10:35:11.069433+0000","last_unstale":"2024-09-13T10:36:31.175963+0000","last_undegraded":"2024-09-13T10:36:31.175963+0000","last_fullsized":"2024-09-13T10:36:31.175963+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:31.175891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:31.175891+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:35.999895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'92","reported_seq":303,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600971+0000","last_change":"2024-09-13T10:35:11.330241+0000","last_active":"2024-09-13T10:36:30.600971+0000","last_peered":"2024-09-13T10:36:30.600971+0000","last_clean":"2024-09-13T10:36:30.600971+0000","last_became_active":"2024-09-13T10:35:11.068333+0000","last_became_peered":"2024-09-13T10:35:11.068333+0000","last_unstale":"2024-09-13T10:36:30.600971+0000","last_undegraded":"2024-09-13T10:36:30.600971+0000","last_fullsized":"2024-09-13T10:36:30.600971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:11.330199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:11.330199+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:38.844085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1932,"ondisk_log_size":1932,"up":42,"acting":42,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1167360,"data_stored":703609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481837,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1167360,"data_stored":703609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743477,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32508,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712644,"statfs":{"total":95995035648,"available":95961747456,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005114,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36668,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708484,"statfs":{"total":95995035648,"available":95957487616,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":995328,"data_stored":606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1372160,"data_stored":904865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117075,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1495040,"data_stored":1031297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313304,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1372160,"data_stored":904865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40644,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704508,"statfs":{"total":95995035648,"available":95953416192,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1495040,"data_stored":1031297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":201248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":201248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:36:39.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T10:36:39.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:39 smithi173 ceph-mon[30864]: pgmap v433: 21 pgs: 21 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:39.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:39 smithi138 ceph-mon[31033]: pgmap v433: 21 pgs: 21 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:39.701 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:40.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:40 smithi060 ceph-mon[27375]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:36:40.381 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T10:36:40.382 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-13T10:36:40.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:40 smithi173 ceph-mon[30864]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:36:40.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:40 smithi138 ceph-mon[31033]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:36:41.119 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-13T10:36:40.001968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1932,"ondisk_log_size":1932,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369312,"kb_used_data":13880,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124572512,"statfs":{"total":1151940427776,"available":1151562252288,"internally_reserved":0,"allocated":14213120,"data_stored":9016819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.893285"},"pg_stats":[{"pgid":"1.14","version":"16'92","reported_seq":102,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:32.551014+0000","last_change":"2024-09-13T10:36:32.551014+0000","last_active":"2024-09-13T10:36:32.551014+0000","last_peered":"2024-09-13T10:36:32.551014+0000","last_clean":"2024-09-13T10:36:32.551014+0000","last_became_active":"2024-09-13T10:35:13.165585+0000","last_became_peered":"2024-09-13T10:35:13.165585+0000","last_unstale":"2024-09-13T10:36:32.551014+0000","last_undegraded":"2024-09-13T10:36:32.551014+0000","last_fullsized":"2024-09-13T10:36:32.551014+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:32.550965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:32.550965+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:55.788595+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'92","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:34.231748+0000","last_change":"2024-09-13T10:36:34.231748+0000","last_active":"2024-09-13T10:36:34.231748+0000","last_peered":"2024-09-13T10:36:34.231748+0000","last_clean":"2024-09-13T10:36:34.231748+0000","last_became_active":"2024-09-13T10:35:13.167887+0000","last_became_peered":"2024-09-13T10:35:13.167887+0000","last_unstale":"2024-09-13T10:36:34.231748+0000","last_undegraded":"2024-09-13T10:36:34.231748+0000","last_fullsized":"2024-09-13T10:36:34.231748+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:34.231682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:34.231682+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:54.889669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11],"acting":[1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'92","reported_seq":91,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.110166+0000","last_change":"2024-09-13T10:35:14.497030+0000","last_active":"2024-09-13T10:36:30.110166+0000","last_peered":"2024-09-13T10:36:30.110166+0000","last_clean":"2024-09-13T10:36:30.110166+0000","last_became_active":"2024-09-13T10:35:13.165149+0000","last_became_peered":"2024-09-13T10:35:13.165149+0000","last_unstale":"2024-09-13T10:36:30.110166+0000","last_undegraded":"2024-09-13T10:36:30.110166+0000","last_fullsized":"2024-09-13T10:36:30.110166+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:14.496969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:14.496969+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:43.515471+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'92","reported_seq":91,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.109086+0000","last_change":"2024-09-13T10:35:14.052060+0000","last_active":"2024-09-13T10:36:30.109086+0000","last_peered":"2024-09-13T10:36:30.109086+0000","last_clean":"2024-09-13T10:36:30.109086+0000","last_became_active":"2024-09-13T10:35:13.167374+0000","last_became_peered":"2024-09-13T10:35:13.167374+0000","last_unstale":"2024-09-13T10:36:30.109086+0000","last_undegraded":"2024-09-13T10:36:30.109086+0000","last_fullsized":"2024-09-13T10:36:30.109086+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:14.052000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:14.052000+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:43.111845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8],"acting":[0,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"16'92","reported_seq":91,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.110127+0000","last_change":"2024-09-13T10:35:13.456223+0000","last_active":"2024-09-13T10:36:30.110127+0000","last_peered":"2024-09-13T10:36:30.110127+0000","last_clean":"2024-09-13T10:36:30.110127+0000","last_became_active":"2024-09-13T10:35:13.163670+0000","last_became_peered":"2024-09-13T10:35:13.163670+0000","last_unstale":"2024-09-13T10:36:30.110127+0000","last_undegraded":"2024-09-13T10:36:30.110127+0000","last_fullsized":"2024-09-13T10:36:30.110127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:13.456177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:13.456177+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:42.956882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'92","reported_seq":130,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.241481+0000","last_change":"2024-09-13T10:35:13.714630+0000","last_active":"2024-09-13T10:36:37.241481+0000","last_peered":"2024-09-13T10:36:37.241481+0000","last_clean":"2024-09-13T10:36:37.241481+0000","last_became_active":"2024-09-13T10:35:13.166768+0000","last_became_peered":"2024-09-13T10:35:13.166768+0000","last_unstale":"2024-09-13T10:36:37.241481+0000","last_undegraded":"2024-09-13T10:36:37.241481+0000","last_fullsized":"2024-09-13T10:36:37.241481+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:13.714577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:13.714577+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:41.321833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'92","reported_seq":104,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.558327+0000","last_change":"2024-09-13T10:36:22.663444+0000","last_active":"2024-09-13T10:36:37.558327+0000","last_peered":"2024-09-13T10:36:37.558327+0000","last_clean":"2024-09-13T10:36:37.558327+0000","last_became_active":"2024-09-13T10:35:13.165018+0000","last_became_peered":"2024-09-13T10:35:13.165018+0000","last_unstale":"2024-09-13T10:36:37.558327+0000","last_undegraded":"2024-09-13T10:36:37.558327+0000","last_fullsized":"2024-09-13T10:36:37.558327+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:22.663361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:22.663361+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:28.652686+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.d","version":"16'92","reported_seq":95,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:38.000819+0000","last_change":"2024-09-13T10:35:13.374342+0000","last_active":"2024-09-13T10:36:38.000819+0000","last_peered":"2024-09-13T10:36:38.000819+0000","last_clean":"2024-09-13T10:36:38.000819+0000","last_became_active":"2024-09-13T10:35:13.164676+0000","last_became_peered":"2024-09-13T10:35:13.164676+0000","last_unstale":"2024-09-13T10:36:38.000819+0000","last_undegraded":"2024-09-13T10:36:38.000819+0000","last_fullsized":"2024-09-13T10:36:38.000819+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:13.374285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:13.374285+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:40.751084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"16'92","reported_seq":108,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:38.000482+0000","last_change":"2024-09-13T10:36:14.842038+0000","last_active":"2024-09-13T10:36:38.000482+0000","last_peered":"2024-09-13T10:36:38.000482+0000","last_clean":"2024-09-13T10:36:38.000482+0000","last_became_active":"2024-09-13T10:35:13.165970+0000","last_became_peered":"2024-09-13T10:35:13.165970+0000","last_unstale":"2024-09-13T10:36:38.000482+0000","last_undegraded":"2024-09-13T10:36:38.000482+0000","last_fullsized":"2024-09-13T10:36:38.000482+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:14.841985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:14.841985+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:16.529239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'92","reported_seq":101,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.460071+0000","last_change":"2024-09-13T10:36:18.690508+0000","last_active":"2024-09-13T10:36:30.460071+0000","last_peered":"2024-09-13T10:36:30.460071+0000","last_clean":"2024-09-13T10:36:30.460071+0000","last_became_active":"2024-09-13T10:35:13.168017+0000","last_became_peered":"2024-09-13T10:35:13.168017+0000","last_unstale":"2024-09-13T10:36:30.460071+0000","last_undegraded":"2024-09-13T10:36:30.460071+0000","last_fullsized":"2024-09-13T10:36:30.460071+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:18.690422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:18.690422+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:25.696673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0],"acting":[11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.8","version":"16'92","reported_seq":130,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.110140+0000","last_change":"2024-09-13T10:36:21.812385+0000","last_active":"2024-09-13T10:36:30.110140+0000","last_peered":"2024-09-13T10:36:30.110140+0000","last_clean":"2024-09-13T10:36:30.110140+0000","last_became_active":"2024-09-13T10:35:01.488976+0000","last_became_peered":"2024-09-13T10:35:01.488976+0000","last_unstale":"2024-09-13T10:36:30.110140+0000","last_undegraded":"2024-09-13T10:36:30.110140+0000","last_fullsized":"2024-09-13T10:36:30.110140+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:21.812316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:21.812316+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:46.770333+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'92","reported_seq":304,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:38.000793+0000","last_change":"2024-09-13T10:36:22.429031+0000","last_active":"2024-09-13T10:36:38.000793+0000","last_peered":"2024-09-13T10:36:38.000793+0000","last_clean":"2024-09-13T10:36:38.000793+0000","last_became_active":"2024-09-13T10:35:01.108329+0000","last_became_peered":"2024-09-13T10:35:01.108329+0000","last_unstale":"2024-09-13T10:36:38.000793+0000","last_undegraded":"2024-09-13T10:36:38.000793+0000","last_fullsized":"2024-09-13T10:36:38.000793+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:22.428973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:22.428973+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:44.863164+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9],"acting":[7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"16'92","reported_seq":121,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.600651+0000","last_change":"2024-09-13T10:36:19.993329+0000","last_active":"2024-09-13T10:36:30.600651+0000","last_peered":"2024-09-13T10:36:30.600651+0000","last_clean":"2024-09-13T10:36:30.600651+0000","last_became_active":"2024-09-13T10:35:01.107379+0000","last_became_peered":"2024-09-13T10:35:01.107379+0000","last_unstale":"2024-09-13T10:36:30.600651+0000","last_undegraded":"2024-09-13T10:36:30.600651+0000","last_fullsized":"2024-09-13T10:36:30.600651+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:19.993270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:19.993270+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:39.451549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6],"acting":[9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"16'92","reported_seq":229,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:30.460070+0000","last_change":"2024-09-13T10:35:11.347909+0000","last_active":"2024-09-13T10:36:30.460070+0000","last_peered":"2024-09-13T10:36:30.460070+0000","last_clean":"2024-09-13T10:36:30.460070+0000","last_became_active":"2024-09-13T10:35:11.067384+0000","last_became_peered":"2024-09-13T10:35:11.067384+0000","last_unstale":"2024-09-13T10:36:30.460070+0000","last_undegraded":"2024-09-13T10:36:30.460070+0000","last_fullsized":"2024-09-13T10:36:30.460070+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:11.347854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:11.347854+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:38.534360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8],"acting":[11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.5","version":"16'92","reported_seq":132,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.300850+0000","last_change":"2024-09-13T10:36:16.503547+0000","last_active":"2024-09-13T10:36:37.300850+0000","last_peered":"2024-09-13T10:36:37.300850+0000","last_clean":"2024-09-13T10:36:37.300850+0000","last_became_active":"2024-09-13T10:35:11.070872+0000","last_became_peered":"2024-09-13T10:35:11.070872+0000","last_unstale":"2024-09-13T10:36:37.300850+0000","last_undegraded":"2024-09-13T10:36:37.300850+0000","last_fullsized":"2024-09-13T10:36:37.300850+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:16.503486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:16.503486+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:17.090008+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2],"acting":[10,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.6","version":"16'92","reported_seq":134,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.300848+0000","last_change":"2024-09-13T10:36:20.447174+0000","last_active":"2024-09-13T10:36:37.300848+0000","last_peered":"2024-09-13T10:36:37.300848+0000","last_clean":"2024-09-13T10:36:37.300848+0000","last_became_active":"2024-09-13T10:35:11.071406+0000","last_became_peered":"2024-09-13T10:35:11.071406+0000","last_unstale":"2024-09-13T10:36:37.300848+0000","last_undegraded":"2024-09-13T10:36:37.300848+0000","last_fullsized":"2024-09-13T10:36:37.300848+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:20.447111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:20.447111+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:21.106848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9],"acting":[10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.7","version":"16'92","reported_seq":133,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.241499+0000","last_change":"2024-09-13T10:36:29.960118+0000","last_active":"2024-09-13T10:36:37.241499+0000","last_peered":"2024-09-13T10:36:37.241499+0000","last_clean":"2024-09-13T10:36:37.241499+0000","last_became_active":"2024-09-13T10:35:11.069951+0000","last_became_peered":"2024-09-13T10:35:11.069951+0000","last_unstale":"2024-09-13T10:36:37.241499+0000","last_undegraded":"2024-09-13T10:36:37.241499+0000","last_fullsized":"2024-09-13T10:36:37.241499+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:29.960042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:29.960042+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:35.190521+0000","scrub_duration":282,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"16'92","reported_seq":132,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:37.300853+0000","last_change":"2024-09-13T10:36:29.286435+0000","last_active":"2024-09-13T10:36:37.300853+0000","last_peered":"2024-09-13T10:36:37.300853+0000","last_clean":"2024-09-13T10:36:37.300853+0000","last_became_active":"2024-09-13T10:35:11.071470+0000","last_became_peered":"2024-09-13T10:35:11.071470+0000","last_unstale":"2024-09-13T10:36:37.300853+0000","last_undegraded":"2024-09-13T10:36:37.300853+0000","last_fullsized":"2024-09-13T10:36:37.300853+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:29.286359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:29.286359+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:58.766540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9],"acting":[10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.2","version":"16'92","reported_seq":315,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:38.000811+0000","last_change":"2024-09-13T10:36:14.382942+0000","last_active":"2024-09-13T10:36:38.000811+0000","last_peered":"2024-09-13T10:36:38.000811+0000","last_clean":"2024-09-13T10:36:38.000811+0000","last_became_active":"2024-09-13T10:35:11.069026+0000","last_became_peered":"2024-09-13T10:35:11.069026+0000","last_unstale":"2024-09-13T10:36:38.000811+0000","last_undegraded":"2024-09-13T10:36:38.000811+0000","last_fullsized":"2024-09-13T10:36:38.000811+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:14.382852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:14.382852+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:21.634070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'92","reported_seq":133,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T10:36:31.175963+0000","last_change":"2024-09-13T10:36:31.175963+0000","last_active":"2024-09-13T10:36:31.175963+0000","last_peered":"2024-09-13T10:36:31.175963+0000","last_clean":"2024-09-13T10:36:31.175963+0000","last_became_active":"2024-09-13T10:35:11.069433+0000","last_became_peered":"2024-09-13T10:35:11.069433+0000","last_unstale":"2024-09-13T10:36:31.175963+0000","last_undegraded":"2024-09-13T10:36:31.175963+0000","last_fullsized":"2024-09-13T10:36:31.175963+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:36:31.175891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:36:31.175891+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:35.999895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'92","reported_seq":305,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T10:36:38.000808+0000","last_change":"2024-09-13T10:35:11.330241+0000","last_active":"2024-09-13T10:36:38.000808+0000","last_peered":"2024-09-13T10:36:38.000808+0000","last_clean":"2024-09-13T10:36:38.000808+0000","last_became_active":"2024-09-13T10:35:11.068333+0000","last_became_peered":"2024-09-13T10:35:11.068333+0000","last_unstale":"2024-09-13T10:36:38.000808+0000","last_undegraded":"2024-09-13T10:36:38.000808+0000","last_fullsized":"2024-09-13T10:36:38.000808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'92","last_scrub_stamp":"2024-09-13T10:35:11.330199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:29:06.842472+0000","last_clean_scrub_stamp":"2024-09-13T10:35:11.330199+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:38.844085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1932,"ondisk_log_size":1932,"up":42,"acting":42,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1167360,"data_stored":703609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":77,"seq":330712481838,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":995328,"data_stored":606724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743477,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32508,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712644,"statfs":{"total":95995035648,"available":95961747456,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005114,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36668,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708484,"statfs":{"total":95995035648,"available":95957487616,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":995328,"data_stored":606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":995328,"data_stored":606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1372160,"data_stored":904865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":995328,"data_stored":606732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1323008,"data_stored":934412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":21,"seq":90194313304,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1372160,"data_stored":904865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40644,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704508,"statfs":{"total":95995035648,"available":95953416192,"internally_reserved":0,"allocated":1167360,"data_stored":703617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1495040,"data_stored":1031297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":201248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":201248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:36:41.121 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-13T10:36:41.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9 shell --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T10:36:41.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:41 smithi060 ceph-mon[27375]: pgmap v434: 21 pgs: 21 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:41.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:41 smithi060 ceph-mon[27375]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:36:41.385 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T10:36:41.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:41 smithi173 ceph-mon[30864]: pgmap v434: 21 pgs: 21 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:41.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:41 smithi173 ceph-mon[30864]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:36:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:41 smithi138 ceph-mon[31033]: pgmap v434: 21 pgs: 21 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T10:36:41.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:41 smithi138 ceph-mon[31033]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T10:36:42.372 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: 1.0 scrub starts 2024-09-13T10:36:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: 1.0 scrub ok 2024-09-13T10:36:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: 1.f scrub starts 2024-09-13T10:36:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: 1.f scrub ok 2024-09-13T10:36:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1154325956' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-13T10:36:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: from='client.? 172.21.15.60:0/1154325956' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-13T10:36:42.373 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:36:42.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:36:42.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:36:42.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 ceph-mon[27375]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:36:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: 1.0 scrub starts 2024-09-13T10:36:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: 1.0 scrub ok 2024-09-13T10:36:42.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: 1.f scrub starts 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: 1.f scrub ok 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1154325956' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: from='client.? 172.21.15.60:0/1154325956' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:36:42.528 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:42 smithi173 ceph-mon[30864]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:36:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: 1.0 scrub starts 2024-09-13T10:36:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: 1.0 scrub ok 2024-09-13T10:36:42.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: 1.f scrub starts 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: 1.f scrub ok 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1154325956' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: from='client.? 172.21.15.60:0/1154325956' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: from='mgr.14150 172.21.15.60:0/2188584602' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-13T10:36:42.601 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:42 smithi138 ceph-mon[31033]: from='mgr.14150 ' entity='mgr.y' 2024-09-13T10:36:42.673 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-13T10:36:42.683 INFO:tasks.cephadm:Teardown begin 2024-09-13T10:36:42.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:36:42.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:36:42.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:36:42.780 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-13T10:36:42.780 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T10:36:42.796 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T10:36:42.814 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T10:36:42.838 INFO:tasks.cephadm:Stopping all daemons... 2024-09-13T10:36:42.838 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-13T10:36:42.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a 2024-09-13T10:36:43.123 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:42 smithi060 systemd[1]: Stopping Ceph mon.a for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:43.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:43 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a[27371]: 2024-09-13T10:36:43.126+0000 7f035f97b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:36:43.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:43 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a[27371]: 2024-09-13T10:36:43.126+0000 7f035f97b640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-13T10:36:43.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:43 smithi060 podman[79356]: 2024-09-13 10:36:43.15760071 +0000 UTC m=+0.132187442 container died 3f6e391b81b9286ea28c7d6811162eba8e0defd39479326945d82e56e74718e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-13T10:36:43.850 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:43 smithi060 podman[79356]: 2024-09-13 10:36:43.568471814 +0000 UTC m=+0.543058546 container cleanup 3f6e391b81b9286ea28c7d6811162eba8e0defd39479326945d82e56e74718e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-13T10:36:43.850 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 13 10:36:43 smithi060 bash[79356]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-a 2024-09-13T10:36:44.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.a.service' 2024-09-13T10:36:44.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:36:44.115 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-13T10:36:44.115 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-13T10:36:44.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.b 2024-09-13T10:36:44.417 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:44 smithi138 systemd[1]: Stopping Ceph mon.b for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:44.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-b[31029]: 2024-09-13T10:36:44.412+0000 7f0c4640d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:36:44.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-b[31029]: 2024-09-13T10:36:44.412+0000 7f0c4640d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-13T10:36:44.850 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:44 smithi138 podman[60622]: 2024-09-13 10:36:44.59613775 +0000 UTC m=+0.286753820 container died 51693fe7572c4caf5f1fcffc7ead30c4413d50a27143fb8ad731a484d49092e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:36:45.215 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:45 smithi138 podman[60622]: 2024-09-13 10:36:45.021167391 +0000 UTC m=+0.711783446 container cleanup 51693fe7572c4caf5f1fcffc7ead30c4413d50a27143fb8ad731a484d49092e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:36:45.215 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:45 smithi138 bash[60622]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-b 2024-09-13T10:36:45.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.b.service' 2024-09-13T10:36:45.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:45 smithi138 podman[60634]: 2024-09-13 10:36:45.214205353 +0000 UTC m=+0.617366792 container remove 51693fe7572c4caf5f1fcffc7ead30c4413d50a27143fb8ad731a484d49092e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-13T10:36:45.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:45 smithi138 systemd[1]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-13T10:36:45.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:45 smithi138 systemd[1]: Stopped Ceph mon.b for 74376cb6-71ba-11ef-bceb-c7b262605968. 2024-09-13T10:36:45.467 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 13 10:36:45 smithi138 systemd[1]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.b.service: Consumed 6.617s CPU time. 2024-09-13T10:36:45.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:36:45.504 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-13T10:36:45.504 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-13T10:36:45.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.c 2024-09-13T10:36:45.892 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:45 smithi173 systemd[1]: Stopping Ceph mon.c for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:46.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-c[30860]: 2024-09-13T10:36:45.889+0000 7f933699a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:36:46.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-c[30860]: 2024-09-13T10:36:45.889+0000 7f933699a640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-13T10:36:46.277 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:46 smithi173 podman[59695]: 2024-09-13 10:36:46.033330958 +0000 UTC m=+0.326617293 container died 034d4332344d4f844d1132e6c2bc6fae2ce3f13bb63f7449cf0f4a4e66ce8d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:36:46.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:46 smithi173 podman[59695]: 2024-09-13 10:36:46.523860357 +0000 UTC m=+0.817146677 container cleanup 034d4332344d4f844d1132e6c2bc6fae2ce3f13bb63f7449cf0f4a4e66ce8d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-13T10:36:46.777 INFO:journalctl@ceph.mon.c.smithi173.stdout:Sep 13 10:36:46 smithi173 bash[59695]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mon-c 2024-09-13T10:36:47.101 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mon.c.service' 2024-09-13T10:36:47.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:36:47.159 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-13T10:36:47.159 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-13T10:36:47.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y 2024-09-13T10:36:47.372 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:36:47 smithi060 systemd[1]: Stopping Ceph mgr.y for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:47.808 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:36:47 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y[27601]: 2024-09-13T10:36:47.464+0000 7f24fb0b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:36:47.808 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:36:47 smithi060 podman[79483]: 2024-09-13 10:36:47.496717714 +0000 UTC m=+0.151337865 container died 6f5f208bdaabd43b2892db1362472301dc8fdf394f01265131ac0767ead501b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-13T10:36:48.122 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:36:47 smithi060 podman[79483]: 2024-09-13 10:36:47.89842313 +0000 UTC m=+0.553043273 container cleanup 6f5f208bdaabd43b2892db1362472301dc8fdf394f01265131ac0767ead501b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d58f73069a8157a9ace12a0247ee3e8497707ea9, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-13T10:36:48.123 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 13 10:36:47 smithi060 bash[79483]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-y 2024-09-13T10:36:48.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-13T10:36:48.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:36:48.363 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-13T10:36:48.363 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-13T10:36:48.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.x 2024-09-13T10:36:48.723 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:36:48 smithi138 systemd[1]: Stopping Ceph mgr.x for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:48.985 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:36:48 smithi138 podman[60747]: 2024-09-13 10:36:48.722015479 +0000 UTC m=+0.158043604 container died 826cd498145f68df66c3d6ec9f51383defd70192e574da28832c4ab9ac6a5b90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:36:49.345 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:36:49 smithi138 podman[60747]: 2024-09-13 10:36:49.084459146 +0000 UTC m=+0.520487261 container cleanup 826cd498145f68df66c3d6ec9f51383defd70192e574da28832c4ab9ac6a5b90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=) 2024-09-13T10:36:49.345 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Sep 13 10:36:49 smithi138 bash[60747]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-mgr-x 2024-09-13T10:36:49.474 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-13T10:36:49.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:36:49.507 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-13T10:36:49.508 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-13T10:36:49.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.0 2024-09-13T10:36:49.837 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:49 smithi060 systemd[1]: Stopping Ceph osd.0 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:50.122 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:49 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0[41584]: 2024-09-13T10:36:49.834+0000 7f5dfbfb1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:36:50.122 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:49 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0[41584]: 2024-09-13T10:36:49.834+0000 7f5dfbfb1640 -1 osd.0 160 *** Got signal Terminated *** 2024-09-13T10:36:50.123 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:49 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0[41584]: 2024-09-13T10:36:49.834+0000 7f5dfbfb1640 -1 osd.0 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:36:55.622 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:55 smithi060 podman[79604]: 2024-09-13 10:36:55.350690698 +0000 UTC m=+5.642528863 container died b146ccd981a08a61feb6ec3c6ca6eee357c0d73bed49f83d92e5d8733555f0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-13T10:36:55.980 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:55 smithi060 podman[79604]: 2024-09-13 10:36:55.769183819 +0000 UTC m=+6.061021982 container cleanup b146ccd981a08a61feb6ec3c6ca6eee357c0d73bed49f83d92e5d8733555f0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-13T10:36:55.980 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:55 smithi060 bash[79604]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0 2024-09-13T10:36:56.372 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:55 smithi060 podman[79620]: 2024-09-13 10:36:55.979030204 +0000 UTC m=+0.623420787 container remove b146ccd981a08a61feb6ec3c6ca6eee357c0d73bed49f83d92e5d8733555f0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:36:56.733 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:56 smithi060 podman[79708]: 2024-09-13 10:36:56.2969997 +0000 UTC m=+0.016643986 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:36:56.734 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:56 smithi060 podman[79708]: 2024-09-13 10:36:56.396510289 +0000 UTC m=+0.116154609 container create e92d78c1e6756d66167742a461b1a79830c35ec19dcbfcea1d858931c226b3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-13T10:36:56.734 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:56 smithi060 podman[79708]: 2024-09-13 10:36:56.632343313 +0000 UTC m=+0.351987587 container init e92d78c1e6756d66167742a461b1a79830c35ec19dcbfcea1d858931c226b3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:36:56.734 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:56 smithi060 podman[79708]: 2024-09-13 10:36:56.635716593 +0000 UTC m=+0.355360868 container start e92d78c1e6756d66167742a461b1a79830c35ec19dcbfcea1d858931c226b3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:36:56.734 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:56 smithi060 podman[79708]: 2024-09-13 10:36:56.732307689 +0000 UTC m=+0.451951975 container attach e92d78c1e6756d66167742a461b1a79830c35ec19dcbfcea1d858931c226b3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:36:57.123 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 13 10:36:56 smithi060 podman[79708]: 2024-09-13 10:36:56.829205352 +0000 UTC m=+0.548849643 container died e92d78c1e6756d66167742a461b1a79830c35ec19dcbfcea1d858931c226b3bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-13T10:36:57.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.0.service' 2024-09-13T10:36:57.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:36:57.496 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-13T10:36:57.496 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-13T10:36:57.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.1 2024-09-13T10:36:57.816 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:36:57 smithi060 systemd[1]: Stopping Ceph osd.1 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:36:58.122 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:36:57 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1[47139]: 2024-09-13T10:36:57.813+0000 7f8548bbe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:36:58.122 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:36:57 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1[47139]: 2024-09-13T10:36:57.813+0000 7f8548bbe640 -1 osd.1 160 *** Got signal Terminated *** 2024-09-13T10:36:58.122 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:36:57 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1[47139]: 2024-09-13T10:36:57.813+0000 7f8548bbe640 -1 osd.1 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:03.449 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:03 smithi060 podman[79812]: 2024-09-13 10:37:03.173869534 +0000 UTC m=+5.448760483 container died a3b90ed61b444fe4377bc45e47b2bfc10a813bc25bcc8168ce4878f1a78c3ac5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T10:37:03.858 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:03 smithi060 podman[79812]: 2024-09-13 10:37:03.547578203 +0000 UTC m=+5.822469159 container cleanup a3b90ed61b444fe4377bc45e47b2bfc10a813bc25bcc8168ce4878f1a78c3ac5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T10:37:03.858 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:03 smithi060 bash[79812]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1 2024-09-13T10:37:04.122 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:03 smithi060 podman[79826]: 2024-09-13 10:37:03.856806218 +0000 UTC m=+0.683303786 container remove a3b90ed61b444fe4377bc45e47b2bfc10a813bc25bcc8168ce4878f1a78c3ac5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T10:37:04.486 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:04 smithi060 podman[79917]: 2024-09-13 10:37:04.076372038 +0000 UTC m=+0.016231424 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:04.486 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:04 smithi060 podman[79917]: 2024-09-13 10:37:04.18262626 +0000 UTC m=+0.122485645 container create 5251b3a233f86ab00b83b52b712a0c33ef772ae2aa1066782a7caf98baed0da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-13T10:37:04.486 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:04 smithi060 podman[79917]: 2024-09-13 10:37:04.40207876 +0000 UTC m=+0.341938157 container init 5251b3a233f86ab00b83b52b712a0c33ef772ae2aa1066782a7caf98baed0da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-13T10:37:04.486 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:04 smithi060 podman[79917]: 2024-09-13 10:37:04.405692881 +0000 UTC m=+0.345552269 container start 5251b3a233f86ab00b83b52b712a0c33ef772ae2aa1066782a7caf98baed0da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T10:37:04.487 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:04 smithi060 podman[79917]: 2024-09-13 10:37:04.484917846 +0000 UTC m=+0.424777248 container attach 5251b3a233f86ab00b83b52b712a0c33ef772ae2aa1066782a7caf98baed0da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:37:04.873 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 13 10:37:04 smithi060 podman[79917]: 2024-09-13 10:37:04.604454914 +0000 UTC m=+0.544314313 container died 5251b3a233f86ab00b83b52b712a0c33ef772ae2aa1066782a7caf98baed0da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-13T10:37:05.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.1.service' 2024-09-13T10:37:05.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:05.218 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-13T10:37:05.218 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-13T10:37:05.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.2 2024-09-13T10:37:05.431 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:05 smithi060 systemd[1]: Stopping Ceph osd.2 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:05.872 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:05 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2[52696]: 2024-09-13T10:37:05.565+0000 7f079ff5b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:05.873 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:05 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2[52696]: 2024-09-13T10:37:05.565+0000 7f079ff5b640 -1 osd.2 160 *** Got signal Terminated *** 2024-09-13T10:37:05.873 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:05 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2[52696]: 2024-09-13T10:37:05.565+0000 7f079ff5b640 -1 osd.2 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:11.252 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:10 smithi060 podman[80022]: 2024-09-13 10:37:10.954692815 +0000 UTC m=+5.509793063 container died 1805ef09d528003a7a29c6dfc2520ee998f7e224f610157db4fe51c65b7784e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-13T10:37:11.619 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:11 smithi060 podman[80022]: 2024-09-13 10:37:11.349480041 +0000 UTC m=+5.904580282 container cleanup 1805ef09d528003a7a29c6dfc2520ee998f7e224f610157db4fe51c65b7784e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:11.619 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:11 smithi060 bash[80022]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2 2024-09-13T10:37:11.872 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:11 smithi060 podman[80047]: 2024-09-13 10:37:11.617562834 +0000 UTC m=+0.656092040 container remove 1805ef09d528003a7a29c6dfc2520ee998f7e224f610157db4fe51c65b7784e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-13T10:37:12.230 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:11 smithi060 podman[80137]: 2024-09-13 10:37:11.951929116 +0000 UTC m=+0.090875975 container create 17720bb9f6037e7c5c3e53f305a3bc3b72a63f26844e06d7bad1d6306bfbe9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:12.231 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:11 smithi060 podman[80137]: 2024-09-13 10:37:11.877333807 +0000 UTC m=+0.016280627 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:12.231 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:12 smithi060 podman[80137]: 2024-09-13 10:37:12.145825625 +0000 UTC m=+0.284772445 container init 17720bb9f6037e7c5c3e53f305a3bc3b72a63f26844e06d7bad1d6306bfbe9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:12.231 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:12 smithi060 podman[80137]: 2024-09-13 10:37:12.14912836 +0000 UTC m=+0.288075178 container start 17720bb9f6037e7c5c3e53f305a3bc3b72a63f26844e06d7bad1d6306bfbe9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-13T10:37:12.231 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:12 smithi060 podman[80137]: 2024-09-13 10:37:12.22907782 +0000 UTC m=+0.368024657 container attach 17720bb9f6037e7c5c3e53f305a3bc3b72a63f26844e06d7bad1d6306bfbe9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-13T10:37:12.622 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:12 smithi060 podman[80137]: 2024-09-13 10:37:12.354759768 +0000 UTC m=+0.493706588 container died 17720bb9f6037e7c5c3e53f305a3bc3b72a63f26844e06d7bad1d6306bfbe9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T10:37:12.979 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 13 10:37:12 smithi060 podman[80160]: 2024-09-13 10:37:12.957926442 +0000 UTC m=+0.597241993 container remove 17720bb9f6037e7c5c3e53f305a3bc3b72a63f26844e06d7bad1d6306bfbe9bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:37:12.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.2.service' 2024-09-13T10:37:13.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:13.032 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-13T10:37:13.033 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-13T10:37:13.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.3 2024-09-13T10:37:13.337 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:13 smithi060 systemd[1]: Stopping Ceph osd.3 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:13.622 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:13 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:37:13.335+0000 7f099e60d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:13.623 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:13 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:37:13.335+0000 7f099e60d640 -1 osd.3 160 *** Got signal Terminated *** 2024-09-13T10:37:13.623 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:13 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:37:13.335+0000 7f099e60d640 -1 osd.3 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:17.122 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:16 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:37:16.749+0000 7f099ac22640 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.948456+0000 front 2024-09-13T10:36:52.948541+0000 (oldest deadline 2024-09-13T10:37:16.349192+0000) 2024-09-13T10:37:17.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:17.042+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:17.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:16.804+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:17.837 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:17 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:17.484+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:18.100 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:17 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:17.835+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:18.122 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:17 smithi060 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3[58365]: 2024-09-13T10:37:17.796+0000 7f099ac22640 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.948456+0000 front 2024-09-13T10:36:52.948541+0000 (oldest deadline 2024-09-13T10:37:16.349192+0000) 2024-09-13T10:37:18.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:18.040+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:18.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:17.819+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:18.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:18 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:18.504+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:18.963 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:18 smithi060 podman[80240]: 2024-09-13 10:37:18.651062021 +0000 UTC m=+5.428025884 container died 53b63a6d04921c67e14b8220bfc64ef4446c1f7104557541faae8a0cfbdc15be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-13T10:37:19.210 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:18 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:18.877+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:19.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:19.055+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:19.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:18.827+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:19.279 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80240]: 2024-09-13 10:37:19.060766061 +0000 UTC m=+5.837729914 container cleanup 53b63a6d04921c67e14b8220bfc64ef4446c1f7104557541faae8a0cfbdc15be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD) 2024-09-13T10:37:19.280 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 bash[80240]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3 2024-09-13T10:37:19.543 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:19 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:19.208+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:19.596 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80256]: 2024-09-13 10:37:19.278270251 +0000 UTC m=+0.617951569 container remove 53b63a6d04921c67e14b8220bfc64ef4446c1f7104557541faae8a0cfbdc15be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-13T10:37:19.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:19 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:19.540+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:19.872 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80345]: 2024-09-13 10:37:19.595640344 +0000 UTC m=+0.111700807 container create 6941423388d578b88e05847f431b396010c7652bc063c99de3fccb33582422af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:19.873 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80345]: 2024-09-13 10:37:19.500709576 +0000 UTC m=+0.016770052 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:19.873 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80345]: 2024-09-13 10:37:19.814686576 +0000 UTC m=+0.330747039 container init 6941423388d578b88e05847f431b396010c7652bc063c99de3fccb33582422af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-13T10:37:19.873 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80345]: 2024-09-13 10:37:19.817945202 +0000 UTC m=+0.334005676 container start 6941423388d578b88e05847f431b396010c7652bc063c99de3fccb33582422af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:20.180 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:19 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:19.888+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:20.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:20.051+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:20.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:19.810+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:20.301 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:19 smithi060 podman[80345]: 2024-09-13 10:37:19.914752062 +0000 UTC m=+0.430812535 container attach 6941423388d578b88e05847f431b396010c7652bc063c99de3fccb33582422af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T10:37:20.302 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 13 10:37:20 smithi060 podman[80345]: 2024-09-13 10:37:20.015342306 +0000 UTC m=+0.531402783 container died 6941423388d578b88e05847f431b396010c7652bc063c99de3fccb33582422af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-13T10:37:20.587 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:20 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:20.178+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:20.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.3.service' 2024-09-13T10:37:20.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:20.632 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-13T10:37:20.633 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-13T10:37:20.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.4 2024-09-13T10:37:20.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:20 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:20.585+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:20.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:20 smithi138 systemd[1]: Stopping Ceph osd.4 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:21.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:20.773+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:21.196 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:20 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:20.919+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:21.197 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:20 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:20.969+0000 7fe72c962640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:21.197 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:20 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:20.969+0000 7fe72c962640 -1 osd.4 160 *** Got signal Terminated *** 2024-09-13T10:37:21.197 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:20 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:20.969+0000 7fe72c962640 -1 osd.4 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:21.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:21 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:21.071+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:21.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:21 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:21.194+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:21.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:21 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:21.615+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:22.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:21 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:21.752+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:22.200 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:21 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:21.883+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:22.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:22 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:22.091+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:22.590 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:22 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:22.198+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:22.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:22 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:22.588+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:23.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:22 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:22.707+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:23.100 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:22 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:22.848+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:23.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:23 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:23.100+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:23.585 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:23 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:23.242+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:23.842 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:23 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:23.583+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:24.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:23 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:23.743+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:24.100 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:23 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:23.840+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:24.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:24 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:24.057+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:24.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:24 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:24.201+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:24.541 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:24 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:24.289+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:24.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:24 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:24.540+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:25.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:24 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:24.773+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:25.027 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:25 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:25.011+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:25.176 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:24 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:24.861+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:25.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:25 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:25.233+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:25.538 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:25 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:25.325+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:25.538 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:25 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:25.175+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:25.850 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:25 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4[35458]: 2024-09-13T10:37:25.537+0000 7fe724f6f640 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:52.060411+0000 front 2024-09-13T10:36:52.060364+0000 (oldest deadline 2024-09-13T10:37:17.260797+0000) 2024-09-13T10:37:26.168 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:25 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:25.848+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:26.220 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:26 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:26.000+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:26.221 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:25 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:25.818+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:26.221 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:25 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:25.818+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:26.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:26 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:26.219+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:26.600 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:26 smithi138 podman[60882]: 2024-09-13 10:37:26.391145686 +0000 UTC m=+5.567501089 container died ed27cc87268b0521d66688f6478d620718149212e70b9c880cc36a551ffa332a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-13T10:37:26.601 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:26 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:26.166+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:26.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:26 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:26.280+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:27.064 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:26 smithi138 podman[60882]: 2024-09-13 10:37:26.753001396 +0000 UTC m=+5.929356794 container cleanup ed27cc87268b0521d66688f6478d620718149212e70b9c880cc36a551ffa332a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T10:37:27.064 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:26 smithi138 bash[60882]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4 2024-09-13T10:37:27.064 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:26 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:26.880+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:27.064 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:26 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:26.880+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:27.204 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:26 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:26.845+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:27.204 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:26 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:26.841+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:27.204 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:26 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:26.998+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:27.350 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60897]: 2024-09-13 10:37:27.062643999 +0000 UTC m=+0.664712719 container remove ed27cc87268b0521d66688f6478d620718149212e70b9c880cc36a551ffa332a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:27.351 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:27 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:27.210+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:27.351 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:27 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:27.269+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:27.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:27 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:27.202+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:27.708 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60987]: 2024-09-13 10:37:27.317602318 +0000 UTC m=+0.016588619 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:27.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60987]: 2024-09-13 10:37:27.430079341 +0000 UTC m=+0.129065631 container create ae6b55daa81359935bd1b8b35b4b13285cf767f1315296403039e27adfbc3358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-13T10:37:27.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60987]: 2024-09-13 10:37:27.632692043 +0000 UTC m=+0.331678329 container init ae6b55daa81359935bd1b8b35b4b13285cf767f1315296403039e27adfbc3358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-13T10:37:27.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60987]: 2024-09-13 10:37:27.636064527 +0000 UTC m=+0.335050822 container start ae6b55daa81359935bd1b8b35b4b13285cf767f1315296403039e27adfbc3358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-13T10:37:27.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60987]: 2024-09-13 10:37:27.707502726 +0000 UTC m=+0.406489026 container attach ae6b55daa81359935bd1b8b35b4b13285cf767f1315296403039e27adfbc3358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True) 2024-09-13T10:37:28.100 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 13 10:37:27 smithi138 podman[60987]: 2024-09-13 10:37:27.824676488 +0000 UTC m=+0.523662792 container died ae6b55daa81359935bd1b8b35b4b13285cf767f1315296403039e27adfbc3358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True) 2024-09-13T10:37:28.101 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:27 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:27.899+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:28.101 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:27 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:27.899+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:28.245 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:27 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:27.834+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:28.245 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:27 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:27.989+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:28.245 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:27 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:27.989+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:28.246 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:27 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:27.808+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:28.395 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:28.255+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:28.396 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:28.300+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:28.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.4.service' 2024-09-13T10:37:28.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:28.454 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-13T10:37:28.455 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-13T10:37:28.455 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.5 2024-09-13T10:37:28.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:28 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:28.242+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:28.662 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:28 smithi138 systemd[1]: Stopping Ceph osd.5 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:28.933 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:28.806+0000 7f81d4b03640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:28.934 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:28.806+0000 7f81d4b03640 -1 osd.5 160 *** Got signal Terminated *** 2024-09-13T10:37:28.934 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:28.806+0000 7f81d4b03640 -1 osd.5 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:29.264 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:28 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:28.853+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:29.265 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:29.007+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:29.265 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:29.007+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:29.265 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:28 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:28.791+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:29.265 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:28 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:28.791+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:29.283 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:28.931+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:29.284 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:28 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:28.931+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:29.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:29.262+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:29.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:29 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:29.281+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:29.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:29 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:29.301+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:30.273 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:29 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:29.967+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:30.274 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:29 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:29.967+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:30.275 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:29.859+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:30.275 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:29.984+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:30.275 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:29.984+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:30.275 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:29.825+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:30.275 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:29 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:29.825+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:30.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:30.273+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:30.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:30 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:30.350+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:30.601 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:30 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:30.272+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:31.251 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:31.000+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:31.252 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:31.000+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:31.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:30.863+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:31.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:30.810+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:31.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:30.810+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:31.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:30.963+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:31.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:30 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:30.963+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:31.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:31.250+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:31.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:31.331+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:31.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:31 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:31.306+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:32.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:31 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:31.910+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:32.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:31 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:31.959+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:32.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:31 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:31.959+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:32.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:31 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:31.845+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:32.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:31 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:31.845+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:32.294 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:31.950+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:32.294 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:31.950+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:32.294 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:31 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:31.950+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.589536+0000 front 2024-09-13T10:37:07.589592+0000 (oldest deadline 2024-09-13T10:37:31.089557+0000) 2024-09-13T10:37:32.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:32 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:32.293+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:32.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:32 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:32.327+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:32.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:32 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:32.284+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:33.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:32 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:32.898+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:33.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:33.000+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:33.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:33.000+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:33.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:32 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:32.797+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:33.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:32 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:32.797+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:33.311 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:32 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:32.922+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:53.687390+0000 front 2024-09-13T10:36:53.687282+0000 (oldest deadline 2024-09-13T10:37:17.187179+0000) 2024-09-13T10:37:33.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:32 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:32.922+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.688047+0000 front 2024-09-13T10:37:00.688178+0000 (oldest deadline 2024-09-13T10:37:25.988098+0000) 2024-09-13T10:37:33.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:32 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5[41343]: 2024-09-13T10:37:32.922+0000 7f81d1118640 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.589536+0000 front 2024-09-13T10:37:07.589592+0000 (oldest deadline 2024-09-13T10:37:31.089557+0000) 2024-09-13T10:37:33.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:33 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:33.312+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:33.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:33 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:33.310+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:33.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:33 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:33.310+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:33.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:33.280+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:33.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:33.280+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:34.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:33.897+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:34.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:34.002+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:34.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:34.002+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:34.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:33.791+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:34.279 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:33 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:33.791+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:34.425 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:34 smithi138 podman[61091]: 2024-09-13 10:37:34.16364161 +0000 UTC m=+5.475199375 container died 263285bcc9e938392c15f2a84a6e2a88575180a945e4c0d9b0247ee77f3c89f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:37:34.426 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:34 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:34.355+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:34.426 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:34 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:34.315+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:34.426 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:34 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:34.315+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:34.757 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:34.283+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:34.758 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:34.283+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:34.825 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:34 smithi138 podman[61091]: 2024-09-13 10:37:34.522825119 +0000 UTC m=+5.834382884 container cleanup 263285bcc9e938392c15f2a84a6e2a88575180a945e4c0d9b0247ee77f3c89f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-13T10:37:34.825 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:34 smithi138 bash[61091]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5 2024-09-13T10:37:35.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:34.880+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:35.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:34.880+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:35.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:34.756+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:35.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:34 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:34.756+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:35.100 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:34 smithi138 podman[61117]: 2024-09-13 10:37:34.824347088 +0000 UTC m=+0.653844878 container remove 263285bcc9e938392c15f2a84a6e2a88575180a945e4c0d9b0247ee77f3c89f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-13T10:37:35.392 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:35 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:35.364+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:35.393 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:35 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:35.364+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:35.393 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:35 smithi138 podman[61208]: 2024-09-13 10:37:35.04143029 +0000 UTC m=+0.016049928 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:35.393 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:35 smithi138 podman[61208]: 2024-09-13 10:37:35.150422755 +0000 UTC m=+0.125042439 container create 0539b56efa1909ed466321c441c11bd416c5a9e4ee671b5bc3165957706c80e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.schema-version=1.0) 2024-09-13T10:37:35.393 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:35 smithi138 podman[61208]: 2024-09-13 10:37:35.377735826 +0000 UTC m=+0.352355465 container init 0539b56efa1909ed466321c441c11bd416c5a9e4ee671b5bc3165957706c80e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:35.393 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:35 smithi138 podman[61208]: 2024-09-13 10:37:35.381387619 +0000 UTC m=+0.356007265 container start 0539b56efa1909ed466321c441c11bd416c5a9e4ee671b5bc3165957706c80e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T10:37:35.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:35.044+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:35.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:35.044+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:35.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:35.244+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:35.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:35.244+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:35.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:35 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:35.390+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:35.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:35 smithi138 podman[61208]: 2024-09-13 10:37:35.477843132 +0000 UTC m=+0.452462786 container attach 0539b56efa1909ed466321c441c11bd416c5a9e4ee671b5bc3165957706c80e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-13T10:37:35.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Sep 13 10:37:35 smithi138 podman[61208]: 2024-09-13 10:37:35.581746251 +0000 UTC m=+0.556365892 container died 0539b56efa1909ed466321c441c11bd416c5a9e4ee671b5bc3165957706c80e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:37:36.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:35.742+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:36.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:35.742+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:36.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:35.870+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:36.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:35 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:35.870+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:36.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.5.service' 2024-09-13T10:37:36.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:36.199 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-13T10:37:36.199 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-13T10:37:36.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.6 2024-09-13T10:37:36.527 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:36.342+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:36.527 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:36.342+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:36.528 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:36 smithi138 systemd[1]: Stopping Ceph osd.6 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:36.528 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:36.372+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:36.528 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:36.372+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:10.486999+0000 front 2024-09-13T10:37:10.487113+0000 (oldest deadline 2024-09-13T10:37:35.787097+0000) 2024-09-13T10:37:36.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:36.048+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:36.529 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:36.048+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:36.529 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:36.263+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:36.529 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:36.263+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:36.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:36.525+0000 7f400ee52640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:36.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:36.525+0000 7f400ee52640 -1 osd.6 160 *** Got signal Terminated *** 2024-09-13T10:37:36.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:36 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:36.525+0000 7f400ee52640 -1 osd.6 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:37.011 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:36.839+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:37.011 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:36.839+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:37.011 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:36.742+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:37.011 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:36 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:36.742+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:37.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:37.009+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:37.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:37.009+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:37.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:37 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:37.342+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:37.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:37 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:37.342+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:10.486999+0000 front 2024-09-13T10:37:10.487113+0000 (oldest deadline 2024-09-13T10:37:35.787097+0000) 2024-09-13T10:37:37.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:37 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:37.319+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:37.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:37 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:37.319+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:37.741 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:37.304+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:37.742 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:37.304+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:38.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:37.740+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:38.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:37.740+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:38.028 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:37.965+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:38.028 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:37.965+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:38.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:37.819+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:38.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:37 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:37.819+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:38.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:38 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:38.362+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:38.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:38 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:38.362+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:10.486999+0000 front 2024-09-13T10:37:10.487113+0000 (oldest deadline 2024-09-13T10:37:35.787097+0000) 2024-09-13T10:37:38.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:38 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:38.330+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:38.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:38 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:38.330+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:38.745 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:38.324+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:38.745 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:38.324+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:39.000 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:38.787+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:39.000 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:38.787+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:39.000 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:38.743+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:39.001 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:38.743+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:39.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:38.997+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:39.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:38 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:38.997+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:39.600 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:39 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:39.380+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:39.601 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:39 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:39.380+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:10.486999+0000 front 2024-09-13T10:37:10.487113+0000 (oldest deadline 2024-09-13T10:37:35.787097+0000) 2024-09-13T10:37:39.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:39 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:39.341+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:39.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:39 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:39.341+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:39.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:39 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:39.341+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:39.715 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:39.326+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:39.716 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:39.326+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:39.980 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:39.837+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:39.981 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:39.837+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:39.981 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:39.714+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:39.981 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:39.714+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:40.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:39.979+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:40.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:39 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:39.979+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:40.716 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:40.295+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:40.716 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:40.295+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:40.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:40 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:40.358+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:40.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:40 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:40.358+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:10.486999+0000 front 2024-09-13T10:37:10.487113+0000 (oldest deadline 2024-09-13T10:37:35.787097+0000) 2024-09-13T10:37:40.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:40 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:40.378+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:40.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:40 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:40.378+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:40.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:40 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:40.378+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:40.992 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:40.791+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:40.992 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:40.791+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:40.992 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:40.713+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:40.992 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:40.713+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:41.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:40.990+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:41.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:40 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:40.990+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:41.731 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:41.321+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:41.732 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:41.321+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:41.732 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:41.321+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:41.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:41 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:41.378+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.286268+0000 front 2024-09-13T10:37:02.286064+0000 (oldest deadline 2024-09-13T10:37:24.586094+0000) 2024-09-13T10:37:41.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:41 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:41.378+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:10.486999+0000 front 2024-09-13T10:37:10.487113+0000 (oldest deadline 2024-09-13T10:37:35.787097+0000) 2024-09-13T10:37:41.851 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:41 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6[47387]: 2024-09-13T10:37:41.378+0000 7f400745f640 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:18.088295+0000 front 2024-09-13T10:37:18.088315+0000 (oldest deadline 2024-09-13T10:37:40.988196+0000) 2024-09-13T10:37:41.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:41 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:41.350+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:41.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:41 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:41.350+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:41.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:41 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:41.350+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:42.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:41.753+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:42.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:41.753+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:42.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:41.730+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:42.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:41 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:41.730+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:42.178 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:41 smithi138 podman[61311]: 2024-09-13 10:37:41.915073114 +0000 UTC m=+5.496475714 container died 369a6f62c1a2438ce74aee6a106bd51000c8ecb6b2a070a7939b0f714689e3f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-13T10:37:42.329 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:42.037+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:42.330 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:42.037+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:42.569 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:42 smithi138 podman[61311]: 2024-09-13 10:37:42.275405949 +0000 UTC m=+5.856808545 container cleanup 369a6f62c1a2438ce74aee6a106bd51000c8ecb6b2a070a7939b0f714689e3f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-13T10:37:42.570 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:42 smithi138 bash[61311]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6 2024-09-13T10:37:42.570 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:42 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:42.373+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:42.570 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:42 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:42.373+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:42.570 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:42 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:42.373+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:42.734 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:42.327+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:42.734 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:42.327+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:42.734 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:42.327+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:42.850 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:42 smithi138 podman[61326]: 2024-09-13 10:37:42.568400176 +0000 UTC m=+0.653462932 container remove 369a6f62c1a2438ce74aee6a106bd51000c8ecb6b2a070a7939b0f714689e3f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-13T10:37:43.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:42.765+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:43.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:42.765+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:43.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:42.765+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:43.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:42.733+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:43.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:42.733+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:43.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:42 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:42.733+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:43.156 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:42 smithi138 podman[61413]: 2024-09-13 10:37:42.877661504 +0000 UTC m=+0.105939694 container create 41c2264911ae6a5f1591ecf92256573492c73c778db8dc67d26c6872886ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-13T10:37:43.156 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:42 smithi138 podman[61413]: 2024-09-13 10:37:42.788055924 +0000 UTC m=+0.016334086 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:43.156 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:43 smithi138 podman[61413]: 2024-09-13 10:37:43.071662177 +0000 UTC m=+0.299940325 container init 41c2264911ae6a5f1591ecf92256573492c73c778db8dc67d26c6872886ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:37:43.156 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:43 smithi138 podman[61413]: 2024-09-13 10:37:43.07493669 +0000 UTC m=+0.303214838 container start 41c2264911ae6a5f1591ecf92256573492c73c778db8dc67d26c6872886ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:43.157 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:43 smithi138 podman[61413]: 2024-09-13 10:37:43.154927611 +0000 UTC m=+0.383205771 container attach 41c2264911ae6a5f1591ecf92256573492c73c778db8dc67d26c6872886ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-13T10:37:43.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:43.050+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:43.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:43.050+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:43.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:43.286+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:43.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:43.286+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:43.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:43.286+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:43.567 INFO:journalctl@ceph.osd.6.smithi138.stdout:Sep 13 10:37:43 smithi138 podman[61413]: 2024-09-13 10:37:43.265469787 +0000 UTC m=+0.493747936 container died 41c2264911ae6a5f1591ecf92256573492c73c778db8dc67d26c6872886ff4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True) 2024-09-13T10:37:43.568 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:43 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:43.345+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:43.568 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:43 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:43.345+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:43.568 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:43 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:43.345+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:43.851 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.6.service' 2024-09-13T10:37:43.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:43.889 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-13T10:37:43.889 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-13T10:37:43.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.7 2024-09-13T10:37:44.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:43.811+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:44.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:43.811+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:44.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:43.811+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:44.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:43.690+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:44.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:43.690+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:44.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:43 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:43.690+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:44.100 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:43 smithi138 systemd[1]: Stopping Ceph osd.7 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:44.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:44.252+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:44.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:44.252+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:44.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:44.252+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:44.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:44.061+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:44.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:44.061+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:44.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:44.061+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:44.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:44.203+0000 7efcb1e92640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:44.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:44.203+0000 7efcb1e92640 -1 osd.7 160 *** Got signal Terminated *** 2024-09-13T10:37:44.600 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:44.203+0000 7efcb1e92640 -1 osd.7 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:44.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:44.305+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:44.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:44.305+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:44.601 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:44 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:44.305+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:45.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:44.824+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:45.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:44.824+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:45.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:44.824+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:45.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:44.700+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:45.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:44.700+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:45.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:44 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:44.700+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:45.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:45.209+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:45.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:45.209+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:45.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:45.209+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:45.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:45.045+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:45.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:45.045+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:45.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:45.045+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:45.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:45 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:45.348+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:45.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:45 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:45.348+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:45.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:45 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:45.348+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:46.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:45.802+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:46.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:45.802+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:46.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:45.802+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:46.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:45.673+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:46.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:45.673+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:46.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:45 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:45.673+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:46.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:46.068+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:46.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:46.068+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:46.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:46.068+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:46.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:46.211+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:46.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:46.211+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:46.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:46.211+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:46.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:46 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:46.396+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:46.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:46 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:46.396+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:46.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:46 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:46.396+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:47.025 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:46.806+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:47.025 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:46.806+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:47.025 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:46.806+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:47.026 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:46.656+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:47.026 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:46.656+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:47.026 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:46 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:46.656+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:47.277 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:47.162+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:47.277 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:47.162+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:47.277 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:47.162+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:47.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:47.023+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:47.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:47.023+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:47.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:47.023+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:47.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:47 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:47.444+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:47.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:47 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:47.444+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:47.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:47 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:47.444+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:48.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:47.851+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:48.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:47.851+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:48.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:47.851+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:48.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:47.639+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:48.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:47.639+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:48.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:47 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:47.639+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:48.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:48.049+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:48.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:48.049+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:48.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:48.049+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:48.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:48.191+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:48.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:48.191+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:48.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:48.191+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:48.529 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:48.191+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:48.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:48 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:48.425+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.540931+0000 front 2024-09-13T10:36:54.541034+0000 (oldest deadline 2024-09-13T10:37:18.640868+0000) 2024-09-13T10:37:48.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:48 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:48.425+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:07.942563+0000 front 2024-09-13T10:37:07.942714+0000 (oldest deadline 2024-09-13T10:37:33.242528+0000) 2024-09-13T10:37:48.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:48 smithi138 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7[53405]: 2024-09-13T10:37:48.425+0000 7efcae4a7640 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:16.743632+0000 front 2024-09-13T10:37:16.743592+0000 (oldest deadline 2024-09-13T10:37:39.043585+0000) 2024-09-13T10:37:49.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:48.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:49.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:48.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:49.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:48.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:49.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:48.612+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:49.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:48.612+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:49.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:48.612+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:49.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:48 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:48.612+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:49.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:49.085+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:49.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:49.085+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:49.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:49.085+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:49.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:49.231+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:49.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:49.231+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:49.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:49.231+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:49.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:49.231+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:49.846 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:49.582+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:49.847 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:49.582+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:49.847 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:49.582+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:49.847 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:49.582+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:49.963 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:49 smithi138 podman[61515]: 2024-09-13 10:37:49.686751649 +0000 UTC m=+5.583049607 container died 9cba1d621c7ed27d3748c8206e6a7667dc5e1fb259778a1df4801864d1969537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-09-13T10:37:50.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:49.845+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:50.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:49.845+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:50.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:49 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:49.845+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:50.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:50.056+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:50.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:50.056+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:50.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:50.056+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:50.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:50.056+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:50.350 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61515]: 2024-09-13 10:37:50.036736036 +0000 UTC m=+5.933034007 container cleanup 9cba1d621c7ed27d3748c8206e6a7667dc5e1fb259778a1df4801864d1969537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:50.350 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 bash[61515]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7 2024-09-13T10:37:50.351 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61530]: 2024-09-13 10:37:50.187488099 +0000 UTC m=+0.491001651 container remove 9cba1d621c7ed27d3748c8206e6a7667dc5e1fb259778a1df4801864d1969537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:37:50.613 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:50.279+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:50.613 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:50.279+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:50.613 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:50.279+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:50.613 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:50.279+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:50.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61615]: 2024-09-13 10:37:50.58813639 +0000 UTC m=+0.109346713 container create 52343714a4bd9375656b6f9a2a9d72316b21a128c68e52f43c1b6a2f22347213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-13T10:37:50.850 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61615]: 2024-09-13 10:37:50.494555411 +0000 UTC m=+0.015765698 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:50.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61615]: 2024-09-13 10:37:50.807301053 +0000 UTC m=+0.328511333 container init 52343714a4bd9375656b6f9a2a9d72316b21a128c68e52f43c1b6a2f22347213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-13T10:37:50.851 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61615]: 2024-09-13 10:37:50.810455229 +0000 UTC m=+0.331665512 container start 52343714a4bd9375656b6f9a2a9d72316b21a128c68e52f43c1b6a2f22347213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:51.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:50.820+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:51.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:50.820+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:51.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:50.820+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:51.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:50.611+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:51.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:50.611+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:51.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:50.611+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:51.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:50 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:50.611+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:51.286 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:50 smithi138 podman[61615]: 2024-09-13 10:37:50.898951153 +0000 UTC m=+0.420161433 container attach 52343714a4bd9375656b6f9a2a9d72316b21a128c68e52f43c1b6a2f22347213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:37:51.286 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:51 smithi138 podman[61615]: 2024-09-13 10:37:51.006022428 +0000 UTC m=+0.527232711 container died 52343714a4bd9375656b6f9a2a9d72316b21a128c68e52f43c1b6a2f22347213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-13T10:37:51.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:51.285+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:51.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:51.285+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:51.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:51.285+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:51.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:51.285+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:51.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.053+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:51.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.053+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:51.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.053+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:51.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.053+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:51.545 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 13 10:37:51 smithi138 podman[61638]: 2024-09-13 10:37:51.527055842 +0000 UTC m=+0.515423371 container remove 52343714a4bd9375656b6f9a2a9d72316b21a128c68e52f43c1b6a2f22347213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:37:51.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.7.service' 2024-09-13T10:37:51.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:51.586 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-13T10:37:51.586 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-13T10:37:51.586 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.8 2024-09-13T10:37:51.906 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 systemd[1]: Stopping Ceph osd.8 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:51.907 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.905+0000 7f9d2f032640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:37:51.907 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.905+0000 7f9d2f032640 -1 osd.8 160 *** Got signal Terminated *** 2024-09-13T10:37:51.907 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:51.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:51.908 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:51.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:51.908 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:51.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:51.908 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:51.822+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:51.908 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:51.598+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:51.908 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:51.598+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:51.909 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:51.598+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:51.909 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:51.599+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:52.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:51 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:51.905+0000 7f9d2f032640 -1 osd.8 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:37:52.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:52.081+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:52.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:52.081+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:52.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:52.081+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:52.278 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:52.081+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:52.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:52.318+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:52.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:52.318+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:52.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:52.318+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:52.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:52.318+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:53.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:52.807+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:53.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:52.807+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:53.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:52.807+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:53.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:52.807+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:53.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:52.573+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:53.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:52.573+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:53.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:52.573+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:53.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:52 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:52.573+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:53.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:53.122+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:53.122+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:53.122+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:53.122+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:53.366+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:53.366+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:53.366+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:53.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:53.366+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:54.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:53.777+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:54.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:53.777+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:54.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:53.777+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:54.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:53.777+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:54.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:53.601+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:54.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:53.601+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:54.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:53.601+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:54.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:53 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:53.601+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:54.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:54.140+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:54.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:54.140+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:54.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:54.140+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:54.528 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:54.140+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:54.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:54.377+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:54.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:54.377+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:54.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:54.377+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:54.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:54.377+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:55.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:54.794+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:55.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:54.794+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:55.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:54.794+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:55.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:54.794+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:55.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:54.794+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:37:55.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:54.560+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:55.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:54.560+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:55.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:54.560+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:55.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:54 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:54.560+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:55.424 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:55.152+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:55.424 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:55.152+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:55.425 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:55.152+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:55.425 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:55.152+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:55.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:55.423+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:55.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:55.423+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:55.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:55.423+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:55.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:55.423+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:55.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:55.577+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:55.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:55.577+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:55.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:55.577+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:55.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:55.577+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:55.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:55.577+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:37:56.113 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:55.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:56.114 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:55.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:56.114 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:55.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:56.114 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:55.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:56.114 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:55 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:55.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:37:56.391 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:56.112+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:54.002579+0000 front 2024-09-13T10:36:54.002551+0000 (oldest deadline 2024-09-13T10:37:16.902610+0000) 2024-09-13T10:37:56.391 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:56.112+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.104241+0000 front 2024-09-13T10:37:02.104253+0000 (oldest deadline 2024-09-13T10:37:27.404173+0000) 2024-09-13T10:37:56.391 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:56.112+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.405639+0000 front 2024-09-13T10:37:14.405614+0000 (oldest deadline 2024-09-13T10:37:43.806063+0000) 2024-09-13T10:37:56.391 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8[35633]: 2024-09-13T10:37:56.112+0000 7f9d2763f640 -1 osd.8 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:23.806804+0000 front 2024-09-13T10:37:23.806717+0000 (oldest deadline 2024-09-13T10:37:49.106537+0000) 2024-09-13T10:37:56.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:56.389+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:56.389+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:56.389+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:56.389+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:56.624+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:56.624+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:56.624+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:56.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:56.624+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:56.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:56.624+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:37:57.245 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:56.832+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:57.245 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:56.832+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:57.246 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:56.832+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:57.246 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:56.832+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:57.246 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:56 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:56.832+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:37:57.527 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:57 smithi173 podman[59818]: 2024-09-13 10:37:57.244356078 +0000 UTC m=+5.462065401 container died c942c08b8604a972cafb8b3459632e464d516fd26a14469b5b7ed10d81daa8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-aclamk-cbt-combined-d58f730, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-13T10:37:57.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:57.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:57.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:57.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:57.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:57.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:57.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:57.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:57.864 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:57 smithi173 podman[59818]: 2024-09-13 10:37:57.788500497 +0000 UTC m=+6.006209819 container cleanup c942c08b8604a972cafb8b3459632e464d516fd26a14469b5b7ed10d81daa8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-aclamk-cbt-combined-d58f730, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-13T10:37:57.864 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:57 smithi173 bash[59818]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8 2024-09-13T10:37:57.864 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:57.619+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:57.864 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:57.619+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:57.865 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:57.619+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:57.865 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:57.619+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:57.865 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:57.619+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:37:58.277 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59832]: 2024-09-13 10:37:58.006690249 +0000 UTC m=+0.757474645 container remove c942c08b8604a972cafb8b3459632e464d516fd26a14469b5b7ed10d81daa8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T10:37:58.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:57.862+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:58.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:57.862+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:58.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:57.862+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:58.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:57.862+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:58.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:57 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:57.862+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:37:58.736 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:58.426+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:58.736 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:58.426+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:58.736 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:58.426+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:58.736 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:58.426+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:58.603+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:58.603+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:58.603+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:58.603+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:58.603+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:37:58.738 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59917]: 2024-09-13 10:37:58.353929895 +0000 UTC m=+0.015824677 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:37:58.738 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59917]: 2024-09-13 10:37:58.474084892 +0000 UTC m=+0.135979659 container create 09519fed6a46a273b16e006dc7360fd5d462a8a7784364d5a08fdbef4b0002f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:37:58.738 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59917]: 2024-09-13 10:37:58.734893766 +0000 UTC m=+0.396788534 container init 09519fed6a46a273b16e006dc7360fd5d462a8a7784364d5a08fdbef4b0002f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-13T10:37:59.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:58.838+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:37:59.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:58.838+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:37:59.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:58.838+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:37:59.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:58.838+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:37:59.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:58 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:58.838+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:37:59.028 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59917]: 2024-09-13 10:37:58.738415047 +0000 UTC m=+0.400309816 container start 09519fed6a46a273b16e006dc7360fd5d462a8a7784364d5a08fdbef4b0002f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD) 2024-09-13T10:37:59.028 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59917]: 2024-09-13 10:37:58.826491898 +0000 UTC m=+0.488386667 container attach 09519fed6a46a273b16e006dc7360fd5d462a8a7784364d5a08fdbef4b0002f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-13T10:37:59.028 INFO:journalctl@ceph.osd.8.smithi173.stdout:Sep 13 10:37:58 smithi173 podman[59917]: 2024-09-13 10:37:58.920341992 +0000 UTC m=+0.582236771 container died 09519fed6a46a273b16e006dc7360fd5d462a8a7784364d5a08fdbef4b0002f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-13T10:37:59.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:59.458+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:37:59.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:59.458+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:37:59.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:59.458+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:37:59.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:37:59.458+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:37:59.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.8.service' 2024-09-13T10:37:59.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:37:59.626 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-13T10:37:59.626 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-13T10:37:59.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.9 2024-09-13T10:37:59.826 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 systemd[1]: Stopping Ceph osd.9 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:37:59.827 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:59.627+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:37:59.827 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:59.627+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:37:59.827 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:59.627+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:37:59.827 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:59.627+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:37:59.827 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:37:59.627+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:00.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.823+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:38:00.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.823+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:38:00.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.823+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:38:00.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.823+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:38:00.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.823+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:38:00.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.941+0000 7f4107e7f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:38:00.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.941+0000 7f4107e7f640 -1 osd.9 160 *** Got signal Terminated *** 2024-09-13T10:38:00.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:37:59 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:37:59.941+0000 7f4107e7f640 -1 osd.9 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:38:00.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:00.462+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:00.462+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:00.462+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:00.462+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:00.462+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:00.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:00.669+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:00.778 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:00.669+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:00.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:00.669+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:00.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:00.669+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:00.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:00.669+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:01.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:00.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:38:01.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:00.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:38:01.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:00.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:38:01.277 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:00.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:38:01.278 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:00 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:00.817+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:38:01.721 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:01.468+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:01.721 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:01.468+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:01.721 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:01.468+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:01.721 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:01.468+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:01.721 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:01.468+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:01.722 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:01.718+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:01.722 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:01.718+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:01.722 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:01.718+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:01.722 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:01.718+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:01.722 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:01.718+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:02.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:01.776+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:38:02.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:01.776+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:38:02.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:01.776+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:38:02.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:01.776+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:38:02.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:01 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:01.776+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:38:02.727 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:02.427+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:02.727 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:02.427+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:02.728 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:02.427+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:02.728 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:02.427+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:02.728 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:02.427+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:03.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:02.725+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:03.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:02.725+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:03.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:02.725+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:03.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:02.725+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:03.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:02.725+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:03.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:02.786+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:38:03.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:02.786+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:38:03.030 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:02.786+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:38:03.030 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:02.786+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:38:03.030 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:02.786+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:38:03.030 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:02 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:02.786+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:38.967867+0000 front 2024-09-13T10:37:38.967833+0000 (oldest deadline 2024-09-13T10:38:02.467761+0000) 2024-09-13T10:38:03.748 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:03.419+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:03.748 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:03.419+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:03.748 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:03.419+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:03.748 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:03.419+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:03.748 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:03.419+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:04.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:03.746+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:04.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:03.746+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:04.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:03.746+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:04.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:03.746+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:04.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:03.746+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:04.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:03.756+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:38:04.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:03.756+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:38:04.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:03.756+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:38:04.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:03.756+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:38:04.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:03.756+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:38:04.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:03 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:03.756+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:38.967867+0000 front 2024-09-13T10:37:38.967833+0000 (oldest deadline 2024-09-13T10:38:02.467761+0000) 2024-09-13T10:38:04.725 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:04.449+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:04.726 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:04.449+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:04.726 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:04.449+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:04.726 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:04.449+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:04.726 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:04.449+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:04.726 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:04.449+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:05.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:04.723+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:05.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:04.723+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:05.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:04.723+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:05.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:04.723+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:05.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:04.723+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:05.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:04.729+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:00.362793+0000 front 2024-09-13T10:37:00.362614+0000 (oldest deadline 2024-09-13T10:37:25.662715+0000) 2024-09-13T10:38:05.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:04.729+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:09.163738+0000 front 2024-09-13T10:37:09.163778+0000 (oldest deadline 2024-09-13T10:37:34.463808+0000) 2024-09-13T10:38:05.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:04.729+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:14.464339+0000 front 2024-09-13T10:37:14.464482+0000 (oldest deadline 2024-09-13T10:37:42.064665+0000) 2024-09-13T10:38:05.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:04.729+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:25.565596+0000 front 2024-09-13T10:37:25.565632+0000 (oldest deadline 2024-09-13T10:37:51.465615+0000) 2024-09-13T10:38:05.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:04.729+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:33.166881+0000 front 2024-09-13T10:37:33.166642+0000 (oldest deadline 2024-09-13T10:37:54.266558+0000) 2024-09-13T10:38:05.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:04 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9[41488]: 2024-09-13T10:38:04.729+0000 7f410048c640 -1 osd.9 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:38.967867+0000 front 2024-09-13T10:37:38.967833+0000 (oldest deadline 2024-09-13T10:38:02.467761+0000) 2024-09-13T10:38:05.701 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:05 smithi173 podman[60019]: 2024-09-13 10:38:05.403174434 +0000 UTC m=+5.560326531 container died 648538563e320cd1f58c4557bde6b119a08c1c5d8974c3966853a3d70aa51107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux ) 2024-09-13T10:38:05.702 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:05.408+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:05.702 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:05.408+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:05.702 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:05.408+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:05.702 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:05.408+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:05.702 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:05.408+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:05.703 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:05.408+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:06.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:05 smithi173 podman[60019]: 2024-09-13 10:38:05.883445421 +0000 UTC m=+6.040597548 container cleanup 648538563e320cd1f58c4557bde6b119a08c1c5d8974c3966853a3d70aa51107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:38:06.027 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:05 smithi173 bash[60019]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9 2024-09-13T10:38:06.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:05.704+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:06.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:05.704+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:06.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:05.704+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:06.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:05.704+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:06.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:05 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:05.704+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:06.389 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60035]: 2024-09-13 10:38:06.134936984 +0000 UTC m=+0.726449037 container remove 648538563e320cd1f58c4557bde6b119a08c1c5d8974c3966853a3d70aa51107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-13T10:38:06.679 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60123]: 2024-09-13 10:38:06.354577629 +0000 UTC m=+0.015980752 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:38:06.679 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60123]: 2024-09-13 10:38:06.479589716 +0000 UTC m=+0.140992829 container create 5ea0590657b14889c73aacb037448b1203774cbd502d71d3e110b3ad46ec9cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-13T10:38:06.679 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:06.386+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:06.679 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:06.386+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:06.680 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:06.386+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:06.680 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:06.386+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:06.680 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:06.386+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:06.680 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:06.386+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:07.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:06.676+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:07.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:06.676+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:07.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:06.676+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:07.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:06.676+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:07.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:06.676+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:07.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:06 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:06.676+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:07.028 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60123]: 2024-09-13 10:38:06.716831149 +0000 UTC m=+0.378234262 container init 5ea0590657b14889c73aacb037448b1203774cbd502d71d3e110b3ad46ec9cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-13T10:38:07.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60123]: 2024-09-13 10:38:06.72004852 +0000 UTC m=+0.381451633 container start 5ea0590657b14889c73aacb037448b1203774cbd502d71d3e110b3ad46ec9cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:38:07.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60123]: 2024-09-13 10:38:06.807166757 +0000 UTC m=+0.468569883 container attach 5ea0590657b14889c73aacb037448b1203774cbd502d71d3e110b3ad46ec9cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-13T10:38:07.029 INFO:journalctl@ceph.osd.9.smithi173.stdout:Sep 13 10:38:06 smithi173 podman[60123]: 2024-09-13 10:38:06.914020038 +0000 UTC m=+0.575423154 container died 5ea0590657b14889c73aacb037448b1203774cbd502d71d3e110b3ad46ec9cc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-aclamk-cbt-combined-d58f730, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-13T10:38:07.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:07.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:07.527 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:07.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:07.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:07.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:07.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:07.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:07.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:07.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:07.528 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:07.412+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:07.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.9.service' 2024-09-13T10:38:07.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:38:07.694 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-13T10:38:07.694 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-13T10:38:07.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.10 2024-09-13T10:38:08.016 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:07 smithi173 systemd[1]: Stopping Ceph osd.10 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:38:08.016 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.013+0000 7f2d7340f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:38:08.017 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.013+0000 7f2d7340f640 -1 osd.10 160 *** Got signal Terminated *** 2024-09-13T10:38:08.017 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.013+0000 7f2d7340f640 -1 osd.10 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:38:08.017 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:07.708+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:08.017 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:07.708+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:08.017 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:07.708+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:08.018 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:07.708+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:08.018 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:07.708+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:08.018 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:07 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:07.708+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:08.749 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.396+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:08.749 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.396+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:08.749 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.396+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:08.749 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.396+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:08.750 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.396+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:08.750 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:08.396+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:09.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:08.745+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:09.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:08.745+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:09.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:08.745+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:09.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:08.745+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:09.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:08.745+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:09.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:08 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:08.745+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:09.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:09.371+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:09.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:09.371+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:09.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:09.371+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:09.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:09.371+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:09.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:09.371+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:09.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:09.371+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:10.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:09.776+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:10.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:09.776+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:10.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:09.776+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:10.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:09.776+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:10.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:09.776+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:10.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:09 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:09.776+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:10.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:10.340+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:10.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:10.340+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:10.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:10.340+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:10.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:10.340+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:10.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:10.340+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:10.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:10.340+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:11.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:10.796+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:11.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:10.796+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:11.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:10.796+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:11.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:10.796+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:11.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:10.796+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:11.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:10 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:10.796+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:11.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:11.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:11.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:11.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:11.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:11.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:11.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:11.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:48.310514+0000 front 2024-09-13T10:37:48.310495+0000 (oldest deadline 2024-09-13T10:38:11.210473+0000) 2024-09-13T10:38:12.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:11.821+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:12.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:11.821+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:12.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:11.821+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:12.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:11.821+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:12.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:11.821+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:12.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:11 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:11.821+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:12.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:36:57.603865+0000 front 2024-09-13T10:36:57.603989+0000 (oldest deadline 2024-09-13T10:37:23.503726+0000) 2024-09-13T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-13T10:37:08.204805+0000 front 2024-09-13T10:37:08.204985+0000 (oldest deadline 2024-09-13T10:37:32.904749+0000) 2024-09-13T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:15.805722+0000 front 2024-09-13T10:37:15.805786+0000 (oldest deadline 2024-09-13T10:37:41.105755+0000) 2024-09-13T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:24.006859+0000 front 2024-09-13T10:37:24.006739+0000 (oldest deadline 2024-09-13T10:37:47.506557+0000) 2024-09-13T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:30.407647+0000 front 2024-09-13T10:37:30.407774+0000 (oldest deadline 2024-09-13T10:37:59.808170+0000) 2024-09-13T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:41.509129+0000 front 2024-09-13T10:37:41.509063+0000 (oldest deadline 2024-09-13T10:38:03.808872+0000) 2024-09-13T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10[47157]: 2024-09-13T10:38:12.378+0000 7f2d6fa24640 -1 osd.10 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:48.310514+0000 front 2024-09-13T10:37:48.310495+0000 (oldest deadline 2024-09-13T10:38:11.210473+0000) 2024-09-13T10:38:13.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:13.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:13.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:13.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:13.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:13.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:13.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:12 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:12.822+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:13.684 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:13 smithi173 podman[60227]: 2024-09-13 10:38:13.397301097 +0000 UTC m=+5.485246248 container died 2491bbfc9d4104f2e19357d24fe4f7555fc0cb5f5b0009c760b9a5f4c3165753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-13T10:38:14.027 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:13 smithi173 podman[60227]: 2024-09-13 10:38:13.847700616 +0000 UTC m=+5.935645763 container cleanup 2491bbfc9d4104f2e19357d24fe4f7555fc0cb5f5b0009c760b9a5f4c3165753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-13T10:38:14.027 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:13 smithi173 bash[60227]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10 2024-09-13T10:38:14.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:14.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:14.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:14.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:14.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:14.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:14.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:13 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:13.836+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:14.511 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60246]: 2024-09-13 10:38:14.157780207 +0000 UTC m=+0.753882663 container remove 2491bbfc9d4104f2e19357d24fe4f7555fc0cb5f5b0009c760b9a5f4c3165753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T10:38:14.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60334]: 2024-09-13 10:38:14.413020656 +0000 UTC m=+0.015669986 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:38:14.777 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60334]: 2024-09-13 10:38:14.516766079 +0000 UTC m=+0.119415413 container create b561f15bbccdb1e398b12d021ad576342009d7031a96630b15e248cb790c2721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T10:38:14.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60334]: 2024-09-13 10:38:14.744528292 +0000 UTC m=+0.347177636 container init b561f15bbccdb1e398b12d021ad576342009d7031a96630b15e248cb790c2721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T10:38:14.778 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60334]: 2024-09-13 10:38:14.747704022 +0000 UTC m=+0.350353361 container start b561f15bbccdb1e398b12d021ad576342009d7031a96630b15e248cb790c2721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-13T10:38:15.215 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60334]: 2024-09-13 10:38:14.819353921 +0000 UTC m=+0.422003263 container attach b561f15bbccdb1e398b12d021ad576342009d7031a96630b15e248cb790c2721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-13T10:38:15.215 INFO:journalctl@ceph.osd.10.smithi173.stdout:Sep 13 10:38:14 smithi173 podman[60334]: 2024-09-13 10:38:14.942846823 +0000 UTC m=+0.545496157 container died b561f15bbccdb1e398b12d021ad576342009d7031a96630b15e248cb790c2721 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-13T10:38:15.215 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:15.215 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:15.216 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:15.216 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:15.216 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:15.216 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:15.216 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:14 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:14.833+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:15.581 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.10.service' 2024-09-13T10:38:15.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:38:15.620 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-13T10:38:15.620 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-13T10:38:15.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.11 2024-09-13T10:38:15.828 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 systemd[1]: Stopping Ceph osd.11 for 74376cb6-71ba-11ef-bceb-c7b262605968... 2024-09-13T10:38:16.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:16.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.826+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.942+0000 7fd507eeb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.942+0000 7fd507eeb640 -1 osd.11 160 *** Got signal Terminated *** 2024-09-13T10:38:16.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:15 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:15.942+0000 7fd507eeb640 -1 osd.11 160 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T10:38:17.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:17.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:17.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:17.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:17.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:17.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:17.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:16 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:16.781+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:18.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:18.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:18.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:18.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:18.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:18.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:18.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:18.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:17 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:17.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.173:6804 osd.8 since back 2024-09-13T10:37:55.769992+0000 front 2024-09-13T10:37:55.770024+0000 (oldest deadline 2024-09-13T10:38:17.470160+0000) 2024-09-13T10:38:19.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:19.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:19.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:19.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:19.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:19.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:19.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:19.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:18 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:18.774+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.173:6804 osd.8 since back 2024-09-13T10:37:55.769992+0000 front 2024-09-13T10:37:55.770024+0000 (oldest deadline 2024-09-13T10:38:17.470160+0000) 2024-09-13T10:38:20.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:20.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:20.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:20.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:20.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:20.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:20.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:20.029 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:19 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:19.741+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.173:6804 osd.8 since back 2024-09-13T10:37:55.769992+0000 front 2024-09-13T10:37:55.770024+0000 (oldest deadline 2024-09-13T10:38:17.470160+0000) 2024-09-13T10:38:21.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-13T10:36:51.462291+0000 front 2024-09-13T10:36:51.462243+0000 (oldest deadline 2024-09-13T10:37:16.762130+0000) 2024-09-13T10:38:21.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-13T10:37:02.663058+0000 front 2024-09-13T10:37:02.663294+0000 (oldest deadline 2024-09-13T10:37:27.962961+0000) 2024-09-13T10:38:21.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-13T10:37:13.164469+0000 front 2024-09-13T10:37:13.164507+0000 (oldest deadline 2024-09-13T10:37:42.565231+0000) 2024-09-13T10:38:21.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6804 osd.4 since back 2024-09-13T10:37:22.565827+0000 front 2024-09-13T10:37:22.565806+0000 (oldest deadline 2024-09-13T10:37:47.865726+0000) 2024-09-13T10:38:21.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-09-13T10:37:32.367578+0000 front 2024-09-13T10:37:32.367511+0000 (oldest deadline 2024-09-13T10:37:54.567844+0000) 2024-09-13T10:38:21.278 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6820 osd.6 since back 2024-09-13T10:37:40.468938+0000 front 2024-09-13T10:37:40.468953+0000 (oldest deadline 2024-09-13T10:38:06.368692+0000) 2024-09-13T10:38:21.279 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.138:6828 osd.7 since back 2024-09-13T10:37:46.369339+0000 front 2024-09-13T10:37:46.369390+0000 (oldest deadline 2024-09-13T10:38:12.269218+0000) 2024-09-13T10:38:21.279 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:20 smithi173 ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11[52623]: 2024-09-13T10:38:20.788+0000 7fd504500640 -1 osd.11 160 heartbeat_check: no reply from 172.21.15.173:6804 osd.8 since back 2024-09-13T10:37:55.769992+0000 front 2024-09-13T10:37:55.770024+0000 (oldest deadline 2024-09-13T10:38:17.470160+0000) 2024-09-13T10:38:21.629 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:21 smithi173 podman[60438]: 2024-09-13 10:38:21.347930063 +0000 UTC m=+5.509802318 container died 8335ad94cf8b55572ba90f50d0a2aead08878b0863ae3d52be2ff78a0cc2da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-aclamk-cbt-combined-d58f730, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-13T10:38:21.971 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:21 smithi173 podman[60438]: 2024-09-13 10:38:21.77633962 +0000 UTC m=+5.938211875 container cleanup 8335ad94cf8b55572ba90f50d0a2aead08878b0863ae3d52be2ff78a0cc2da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-aclamk-cbt-combined-d58f730, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-13T10:38:21.971 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:21 smithi173 bash[60438]: ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11 2024-09-13T10:38:22.277 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:21 smithi173 podman[60455]: 2024-09-13 10:38:21.969600546 +0000 UTC m=+0.611938713 container remove 8335ad94cf8b55572ba90f50d0a2aead08878b0863ae3d52be2ff78a0cc2da9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730, ceph=True) 2024-09-13T10:38:22.665 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:22 smithi173 podman[60545]: 2024-09-13 10:38:22.289485492 +0000 UTC m=+0.015836924 image pull fec07016a2b0bf363d0b69240081049b65cf13f1c44f6a7196cf8309aeb428cd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f 2024-09-13T10:38:22.665 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:22 smithi173 podman[60545]: 2024-09-13 10:38:22.395268441 +0000 UTC m=+0.121619864 container create 6571085c5e1089858411a836ac58ac8224a5c9a261f6a786d278d16b07682358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-13T10:38:22.666 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:22 smithi173 podman[60545]: 2024-09-13 10:38:22.589241392 +0000 UTC m=+0.315592814 container init 6571085c5e1089858411a836ac58ac8224a5c9a261f6a786d278d16b07682358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-13T10:38:22.666 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:22 smithi173 podman[60545]: 2024-09-13 10:38:22.592557554 +0000 UTC m=+0.318908978 container start 6571085c5e1089858411a836ac58ac8224a5c9a261f6a786d278d16b07682358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-13T10:38:22.666 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:22 smithi173 podman[60545]: 2024-09-13 10:38:22.66430528 +0000 UTC m=+0.390656704 container attach 6571085c5e1089858411a836ac58ac8224a5c9a261f6a786d278d16b07682358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=wip-aclamk-cbt-combined-d58f730, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-13T10:38:23.027 INFO:journalctl@ceph.osd.11.smithi173.stdout:Sep 13 10:38:22 smithi173 podman[60545]: 2024-09-13 10:38:22.776524148 +0000 UTC m=+0.502875575 container died 6571085c5e1089858411a836ac58ac8224a5c9a261f6a786d278d16b07682358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:914e971332a58dba507db28f0971f6f283c67edecc2d9a6537eea399760b8c1f, name=ceph-74376cb6-71ba-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-aclamk-cbt-combined-d58f730) 2024-09-13T10:38:23.425 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-74376cb6-71ba-11ef-bceb-c7b262605968@osd.11.service' 2024-09-13T10:38:23.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T10:38:23.463 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-13T10:38:23.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T10:38:23.677 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:38:25.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T10:38:25.606 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:38:27.358 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T10:38:27.559 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:38:29.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:38:29.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:38:29.143 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T10:38:29.170 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-13T10:38:29.172 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi060/crash 2024-09-13T10:38:29.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash -- . 2024-09-13T10:38:29.214 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T10:38:29.214 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2024-09-13T10:38:29.216 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi138/crash 2024-09-13T10:38:29.217 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash -- . 2024-09-13T10:38:29.286 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T10:38:29.287 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-09-13T10:38:29.289 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi173/crash 2024-09-13T10:38:29.290 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash -- . 2024-09-13T10:38:29.325 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T10:38:29.325 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-09-13T10:38:29.326 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-13T10:38:29.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(REQUEST_STUCK\)' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-13T10:38:29.367 INFO:tasks.cephadm:Compressing logs... 2024-09-13T10:38:29.367 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:38:29.410 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:38:29.412 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:38:29.435 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-13T10:38:29.435 INFO:teuthology.orchestra.run.smithi060.stderr:: No such file or directory 2024-09-13T10:38:29.435 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-13T10:38:29.436 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-13T10:38:29.436 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-13T10:38:29.437 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-13T10:38:29.437 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-13T10:38:29.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-13T10:38:29.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip 82.6% -5 -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log.gz --verbose 2024-09-13T10:38:29.439 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-13T10:38:29.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log 2024-09-13T10:38:29.442 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:: No such file or directory 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.9.log: /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-13T10:38:29.443 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-13T10:38:29.444 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log 2024-09-13T10:38:29.444 INFO:teuthology.orchestra.run.smithi173.stderr: 82.3% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-13T10:38:29.444 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-13T10:38:29.444 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-13T10:38:29.444 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.2% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-13T10:38:29.444 INFO:teuthology.orchestra.run.smithi173.stderr: 85.8% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-13T10:38:29.445 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-13T10:38:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-13T10:38:29.446 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mgr.y.log: /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log: 90.0% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-13T10:38:29.446 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-13T10:38:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-13T10:38:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log: -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr: 82.4% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-13T10:38:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr: 90.2%/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mgr.x.log: -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr: 85.8% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr: 91.1% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-13T10:38:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.b.log: 91.3% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-13T10:38:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-13T10:38:29.452 INFO:teuthology.orchestra.run.smithi173.stderr: 91.3% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-13T10:38:29.453 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-13T10:38:29.453 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-13T10:38:29.454 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 92.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-13T10:38:29.557 INFO:teuthology.orchestra.run.smithi173.stderr: 92.3% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-13T10:38:29.573 INFO:teuthology.orchestra.run.smithi173.stderr: 92.3% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-13T10:38:29.584 INFO:teuthology.orchestra.run.smithi173.stderr: 91.8% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-13T10:38:29.591 INFO:teuthology.orchestra.run.smithi138.stderr: 92.4% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-13T10:38:29.593 INFO:teuthology.orchestra.run.smithi138.stderr: 92.0% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-13T10:38:29.597 INFO:teuthology.orchestra.run.smithi138.stderr: 91.6% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-13T10:38:29.601 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-13T10:38:29.604 INFO:teuthology.orchestra.run.smithi173.stderr: 92.4% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-13T10:38:29.611 INFO:teuthology.orchestra.run.smithi060.stderr: 92.0% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-13T10:38:29.626 INFO:teuthology.orchestra.run.smithi060.stderr: 91.9% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-13T10:38:29.639 INFO:teuthology.orchestra.run.smithi138.stderr: 92.6% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-13T10:38:29.643 INFO:teuthology.orchestra.run.smithi060.stderr: 92.4% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-13T10:38:29.664 INFO:teuthology.orchestra.run.smithi138.stderr: 91.5% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-13T10:38:29.666 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-09-13T10:38:29.666 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m0.239s 2024-09-13T10:38:29.666 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m0.879s 2024-09-13T10:38:29.666 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.047s 2024-09-13T10:38:29.713 INFO:teuthology.orchestra.run.smithi173.stderr: 94.1% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-13T10:38:29.714 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-09-13T10:38:29.714 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m0.287s 2024-09-13T10:38:29.715 INFO:teuthology.orchestra.run.smithi173.stderr:user 0m0.827s 2024-09-13T10:38:29.715 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m0.053s 2024-09-13T10:38:29.850 INFO:teuthology.orchestra.run.smithi060.stderr: 90.5% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-13T10:38:30.107 INFO:teuthology.orchestra.run.smithi060.stderr: 95.5% -- replaced with /var/log/ceph/74376cb6-71ba-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-13T10:38:30.109 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-13T10:38:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m0.685s 2024-09-13T10:38:30.109 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m1.778s 2024-09-13T10:38:30.110 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m0.080s 2024-09-13T10:38:30.110 INFO:tasks.cephadm:Archiving logs... 2024-09-13T10:38:30.110 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi060/log 2024-09-13T10:38:30.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:38:30.425 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi138/log 2024-09-13T10:38:30.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:38:30.575 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi173/log 2024-09-13T10:38:30.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:38:30.739 INFO:tasks.cephadm:Removing cluster... 2024-09-13T10:38:30.739 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --force 2024-09-13T10:38:30.939 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:38:31.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --force 2024-09-13T10:38:31.401 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:38:31.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 74376cb6-71ba-11ef-bceb-c7b262605968 --force 2024-09-13T10:38:31.854 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 74376cb6-71ba-11ef-bceb-c7b262605968 2024-09-13T10:38:32.113 INFO:tasks.cephadm:Removing cephadm ... 2024-09-13T10:38:32.113 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T10:38:32.133 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T10:38:32.150 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T10:38:32.166 INFO:tasks.cephadm:Teardown complete 2024-09-13T10:38:32.166 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T10:38:32.176 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T10:38:32.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:38:32.178 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:38:32.192 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:38:32.307 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T10:38:32.315 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T10:38:32.316 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:38:32.318 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:38:32.321 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:38:32.333 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-13T10:38:32.334 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-13T10:38:32.335 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-09-13T10:38:32.338 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T10:38:32.339 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-13T10:38:32.339 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 -210us[ -199us] +/- 50ms 2024-09-13T10:38:32.339 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 +359us[ +359us] +/- 76ms 2024-09-13T10:38:32.339 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -11us[ -11us] +/- 67ms 2024-09-13T10:38:32.339 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:38:32.341 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T10:38:32.341 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-09-13T10:38:32.341 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -199us[ -199us] +/- 50ms 2024-09-13T10:38:32.341 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 +352us[ +352us] +/- 76ms 2024-09-13T10:38:32.341 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -13us[ -12us] +/- 67ms 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -215us[ -206us] +/- 50ms 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 +358us[ +358us] +/- 76ms 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -28us[ -19us] +/- 67ms 2024-09-13T10:38:32.342 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T10:38:32.343 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T10:38:32.352 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T10:38:32.353 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T10:38:32.375 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T10:38:32.463 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T10:38:32.531 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T10:38:32.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T10:38:32.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:38:32.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T10:38:32.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:38:32.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-13T10:38:32.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:38:32.688 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T10:38:32.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T10:38:32.733 INFO:teuthology.task.internal:Duration was 1133.331711 seconds 2024-09-13T10:38:32.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T10:38:32.767 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T10:38:32.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:38:32.771 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:38:32.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:38:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T10:38:32.811 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T10:38:32.814 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T10:38:33.323 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T10:38:33.324 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T10:38:33.324 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:38:33.351 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-09-13T10:38:33.352 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:38:33.377 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-09-13T10:38:33.378 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:38:33.403 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T10:38:33.404 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:38:33.406 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:38:33.420 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:38:33.470 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T10:38:33.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:38:33.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:38:33.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:38:33.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T10:38:33.871 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T10:38:33.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:38:33.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:38:33.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:38:33.947 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T10:38:33.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:38:33.960 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:38:33.965 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:38:33.981 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-09-13T10:38:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-09-13T10:38:34.011 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-09-13T10:38:34.057 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:38:34.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:38:34.080 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:38:34.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:38:34.095 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:38:34.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:38:34.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T10:38:34.121 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T10:38:34.122 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi060 2024-09-13T10:38:34.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:38:34.163 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi138 2024-09-13T10:38:34.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:38:34.199 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903352/remote/smithi173 2024-09-13T10:38:34.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:38:34.234 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T10:38:34.234 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:38:34.237 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:38:34.239 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:38:34.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T10:38:34.295 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T10:38:34.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T10:38:34.317 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T10:38:34.317 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:38:34.320 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:38:34.322 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:38:34.333 INFO:teuthology.orchestra.run.smithi060.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:38 /home/ubuntu/cephtest 2024-09-13T10:38:34.335 INFO:teuthology.orchestra.run.smithi138.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:38 /home/ubuntu/cephtest 2024-09-13T10:38:34.338 INFO:teuthology.orchestra.run.smithi173.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:38 /home/ubuntu/cephtest 2024-09-13T10:38:34.340 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T10:38:34.350 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T10:38:34.421 INFO:teuthology.run:Summary data: description: rados/thrash-old-clients/{0-distro$/{centos_9.stream} 0-size-min-size-overrides/2-size-2-min-size 1-install/reef backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} duration: 1133.331710577011 failure_reason: 'Command failed on smithi173 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror''"' flavor: default owner: scheduled_akupczyk@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5d8efaf651ad44dd8898cd78302af9d0 status: fail success: false 2024-09-13T10:38:34.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T10:38:34.499 INFO:teuthology.run:FAIL